Add calibration tooling (calibrate.py) with tests; expand characterization/comparison analysis and docs
This commit is contained in:
parent
c2bb3efa93
commit
f142b8e6ff
13 changed files with 2368 additions and 97 deletions
32
NOTES.md
32
NOTES.md
|
|
@ -29,7 +29,12 @@ are no longer right.
|
||||||
| `~84,333 / cc / 3` for the sensor ceiling | fitting overhead separately | `1 / (3 × (cc/90,000 + 68.7 µs))`. The naive constant is not constant — it drifts from 84,429 at cc=100 to 89,191 at cc=1200 |
|
| `~84,333 / cc / 3` for the sensor ceiling | fitting overhead separately | `1 / (3 × (cc/90,000 + 68.7 µs))`. The naive constant is not constant — it drifts from 84,429 at cc=100 to 89,191 at cc=1200 |
|
||||||
| "3.3 V vs 3.0 V — indistinguishable" | interleaved A/B at two cycle counts, 2026-08-23 | Indistinguishable in *noise*, yes. But the **scale differs by 7.1%** and the **sample rate by 4.94%**. The earlier entry could not see it because the sensor had moved and the two effects were confounded |
|
| "3.3 V vs 3.0 V — indistinguishable" | interleaved A/B at two cycle counts, 2026-08-23 | Indistinguishable in *noise*, yes. But the **scale differs by 7.1%** and the **sample rate by 4.94%**. The earlier entry could not see it because the sensor had moved and the two effects were confounded |
|
||||||
| "60 Hz is 25× the noise floor and dominates every sd figure" | the same session, at 285–299 Hz | True at ~35 Hz where it folds to 9 Hz. At the default rate mains is resolved in band at only **1.7–1.8×** the floor. At cc=400 the dominant line is not mains at all but an **fs/4 artefact** |
|
| "60 Hz is 25× the noise floor and dominates every sd figure" | the same session, at 285–299 Hz | True at ~35 Hz where it folds to 9 Hz. At the default rate mains is resolved in band at only **1.7–1.8×** the floor. At cc=400 the dominant line is not mains at all but an **fs/4 artefact** |
|
||||||
| "68.7 µs per-axis overhead" | solving the model per supply | Right in order, wrong in value for this unit: **40.6 µs at 3.006 V, 38.1 µs at 3.353 V**. The 90,000 counts/s divisor holds up; the overhead does not, and it moves with the supply |
|
| "68.7 µs per-axis overhead" | solving the model per supply | Right in order, wrong in value for this unit: **40.6 µs at 3.006 V, 38.1 µs at 3.353 V**. And the divisor does not hold either — solving both freely gives **88,546 counts/s** at 3.006 V and **92,889** at 3.353 V. Holding it at 90,000 and solving for the overhead alone gives 58.9 µs at cc=100 and 113.6 µs at cc=400, fitting neither. Both terms are per-unit and supply-dependent; `calibrate.py` carries them |
|
||||||
|
| "`× (1 + rate_error)` corrects gain, cutting the spread to 3.1%" | referencing the correction to a measured oscillator | **The 3.1% was two errors cancelling.** `rate_error` is measured against the *nominal* model, whose error is cycle-count dependent (+0.84% at cc=100, −0.99% at cc=400, same supply), and that partly cancelled the cycle-count gain difference. Referenced properly the figure is **4.9%**. The matched-cycle-count contrasts are unaffected either way |
|
||||||
|
| "gain ∝ V^−0.65, i.e. rate^−1.47; mechanism unresolved" | freeing the gain offset | **The 1.47 was absorbing a gain-offset error.** Table 3-1's `0.3671·cc + 1.5` implies gain ∝ (cc + 4.086) counts; measured on this unit it is 0.90 (LDO) and −0.09 (3V3). With the offset free, an exponent of exactly **1** is fully consistent — which is what an oscillator-clocked collection interval predicts. Correcting both leaves **2.48%**, inside the 2.3–3.1% movement floor |
|
||||||
|
| "the per-axis overhead is constant in counts (0.7σ) and changes as a time (2.8σ)" | propagating the thermal drift properly | Those σ came from a half-drift error bar chosen without justification. Propagated, the overhead is 3.60 ± 0.30 counts against 3.54 ± 0.15, and neither reading is excluded. The clean test is the supply rate ratio across cycle counts: oscillator-clocked sits **0.3σ** from it, fixed-time **1.2σ**. Favoured, not proven — drift is the limit |
|
||||||
|
| "the within-run drift could be used to correct gain in post-processing" | the coupling test, then σ_y(τ) | **Closed — do not reopen.** Gain does not follow the within-run rate: the measured slopes are **+0.08 to +0.35** where gain ∝ 1/f demands **−1**. The mechanism is that gain is set by the *ratio* of the MagI2C internal clock to f_LR, both on the same die, so a thermal shift is common-mode and divides out. That is what PNI's "stable over temperature" claim in §2 actually protects, and it is why 1241 ppm of rate drift produces no measurable gain drift. The drift is real and well resolved; it simply does not reach the gain |
|
||||||
|
| "the rate measurement is limited by host read jitter, so a longer window is always better" | σ_y(τ) on the four captures | Wrong in both halves. White jitter predicts σ/rate = √12·σ_t/(√n·T), or sub-ppm at 30 s; measured, σ_y bottoms out at **8–14 ppm around τ = 2–10 s** and then *rises*. The floor is correlated host-side noise, 10–30× the naive figure, and past ~10 s a longer window makes the estimate worse, not better — it starts measuring the drift instead of the rate |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -487,6 +492,31 @@ fs/4 and fs/2 harmonics. That excludes a fixed digital pattern (which would be
|
||||||
1.000, +2.5σ) and external pickup (0.935, +3.3σ), and sits between V and V²,
|
1.000, +2.5σ) and external pickup (0.935, +3.3σ), and sits between V and V²,
|
||||||
which two rail voltages cannot separate.
|
which two rail voltages cannot separate.
|
||||||
|
|
||||||
|
**Every capture drifts through its own thermal warm-up** — 186 to 1241 ppm,
|
||||||
|
monotonically downward, measured by re-fitting the rate on sub-windows
|
||||||
|
(`calibrate.rate_stability`). That is the error bar under every timing figure
|
||||||
|
here, and the reason the collection-interval mechanism is favoured rather than
|
||||||
|
established. Let the rig settle before a run that matters.
|
||||||
|
|
||||||
|
**That drift is the sample rate, not the field**, and the distinction is worth
|
||||||
|
keeping because the natural explanations for "drift" on a magnetometer are
|
||||||
|
geomagnetic. `dt_true` is fitted from the host clock against sample index, so
|
||||||
|
nothing magnetic can reach it, and the pattern confirms it: every run starts high
|
||||||
|
and ends low, then the *next* run starts high again. A diurnal trend would run
|
||||||
|
continuously across the session; this resets at each `logger.py` invocation.
|
||||||
|
|
||||||
|
The **field** drift within a run is separate and probably is diurnal: |B| moves
|
||||||
|
−1.4 to −1.8 nT/min in three of the four runs, in a consistent direction, which
|
||||||
|
is the right order for Sq variation. It cannot be the 2.48% residual — that is
|
||||||
|
1,166 nT against ~40 nT of diurnal across the whole 26-minute session.
|
||||||
|
|
||||||
|
**The sensor moved during a run, not only between them.** LDO/cc400 rotates
|
||||||
|
**0.49°** across its own run, ten times the quietest, and that is the same run
|
||||||
|
carrying 485 nT of Z drift at nearly constant |B| — which is rotation, not field.
|
||||||
|
The rig was not restrained and had to be handled to move the 3 V jumper between
|
||||||
|
runs, so this is expected rather than surprising; it is why the residual is
|
||||||
|
attributed to movement.
|
||||||
|
|
||||||
**Filtering and decimating a fast capture matches sampling slowly**, which the
|
**Filtering and decimating a fast capture matches sampling slowly**, which the
|
||||||
README had been asserting without evidence. Decimating a cc=100 capture by 4
|
README had been asserting without evidence. Decimating a cc=100 capture by 4
|
||||||
changes its own broadband floor by **+0.3 to +0.5%** — a comparison with no
|
changes its own broadband floor by **+0.3 to +0.5%** — a comparison with no
|
||||||
|
|
|
||||||
96
README.md
96
README.md
|
|
@ -95,21 +95,24 @@ differ"*). 3.353 V is comfortably in range, but the gain fit behind
|
||||||
extrapolation off the calibration point. **This has now been measured**, on
|
extrapolation off the calibration point. **This has now been measured**, on
|
||||||
interleaved captures at 3.006 V and 3.353 V:
|
interleaved captures at 3.006 V and 3.353 V:
|
||||||
|
|
||||||
| | effect of +0.347 V | as a power of V |
|
| | effect of +0.347 V |
|
||||||
|---|---|---|
|
|---|---|
|
||||||
| **Sample rate** | **+4.94%** (repeats to 0.04%) | V^+0.44 |
|
| **Sample rate** | **+4.94%** (repeats to 0.04% across two cycle counts) |
|
||||||
| **Total field \|B\|** | **−7.1%**, about −3,245 nT | V^−0.65 |
|
| **Total field \|B\|** | **−7.1%**, about −3,245 nT |
|
||||||
| Broadband noise | none resolvable, ~4% fractionally | — |
|
| Broadband noise | none resolvable, ~4% fractionally |
|
||||||
|
|
||||||
So **supply quality does not set the noise floor, but supply *voltage* sets the
|
So **supply quality does not set the noise floor, but supply *voltage* sets the
|
||||||
scale**, at roughly −20 %/V. A ±1% regulator at 3.0 V holds scale to ~0.2%; a
|
scale**, at roughly −20 %/V. A ±1% regulator at 3.0 V holds scale to ~0.2%; a
|
||||||
USB-derived rail free to move ±5% holds it to ~1%, on top of the ~7% fixed offset
|
USB-derived rail free to move ±5% holds it to ~1%, on top of the ~7% fixed offset
|
||||||
from sitting off the calibration point.
|
from sitting off the calibration point.
|
||||||
|
|
||||||
The chip's oscillator carries the same dependence, which is useful: the
|
Both effects are the same thing seen twice. The chip's measurement interval is
|
||||||
calibrated period every capture already records tracks the rail without a
|
clocked by its own oscillator — the per-axis overhead is a fixed ~3.6 *counts*,
|
||||||
voltmeter, and correcting |B| by the measured rate removes most of the offset.
|
not a fixed time — so the same number of counts is collected over a window whose
|
||||||
Full analysis, including what is *not* settled, in
|
duration goes as 1/f. Gain follows integration time, giving `gain ∝ (cc + n)/f`
|
||||||
|
with **no free exponent**. That makes the calibrated period every capture already
|
||||||
|
records a proxy for the rail, correctable without a voltmeter. Full analysis,
|
||||||
|
including what is *not* settled, in
|
||||||
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
|
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
|
||||||
[NOTES.md](NOTES.md) covers what the manual does and does not say about ripple.
|
[NOTES.md](NOTES.md) covers what the manual does and does not say about ripple.
|
||||||
|
|
||||||
|
|
@ -234,6 +237,19 @@ host clock. Both appear in the capture header.
|
||||||
several captures means anything. `--spectrogram`, `--overlap`, `--max-freq`
|
several captures means anything. `--spectrogram`, `--overlap`, `--max-freq`
|
||||||
and `--colormap` tune it; the default `turbo` makes narrow lines legible,
|
and `--colormap` tune it; the default `turbo` makes narrow lines legible,
|
||||||
`viridis` is the lightness-monotonic option.
|
`viridis` is the lightness-monotonic option.
|
||||||
|
Two further figures come out beside it:
|
||||||
|
`_drift.png` — the chip's oscillator measured against the host clock, as rate
|
||||||
|
against elapsed time with error bars, and beside it the two-sample deviation
|
||||||
|
σ_y(τ) that supplies those bars. The minimum of that curve is the best the
|
||||||
|
rate can be known and the averaging time worth using; the branch rising ∝ τ to
|
||||||
|
its right is real drift, not measurement noise. These parts run 186–1241 ppm
|
||||||
|
of warm-up against a floor near 10 ppm at τ ≈ 2–10 s.
|
||||||
|
`_timebase.png` — for each detected line, its coherent amplitude computed on
|
||||||
|
the chip's sample grid and again on the host clock. A line at a fixed
|
||||||
|
frequency sharpens on the host clock; one locked to the sampling is wrecked by
|
||||||
|
it. That settles **from a single capture** what `compare.py` otherwise needs
|
||||||
|
two different rates to decide. Diagnostic only: no other figure uses the host
|
||||||
|
time base, because neither choice is right for both kinds of line.
|
||||||
- **`compare.py`** — everything cross-capture. Reports noise **fractionally**
|
- **`compare.py`** — everything cross-capture. Reports noise **fractionally**
|
||||||
(ppm of |B|), because a gain change carries the noise with it and an absolute
|
(ppm of |B|), because a gain change carries the noise with it and an absolute
|
||||||
comparison reads a pure scale change as a noise difference. Also: one common
|
comparison reads a pure scale change as a noise difference. Also: one common
|
||||||
|
|
@ -243,13 +259,67 @@ host clock. Both appear in the capture header.
|
||||||
correction. Label captures `CONDITION/variant=path` — the part before the `/`
|
correction. Label captures `CONDITION/variant=path` — the part before the `/`
|
||||||
is the thing under test, and `--supply CONDITION=VOLTS` attaches a rail to it.
|
is the thing under test, and `--supply CONDITION=VOLTS` attaches a rail to it.
|
||||||
- **`sweep.py`** — runs `logger.py` once per target rate and tabulates what each
|
- **`sweep.py`** — runs `logger.py` once per target rate and tabulates what each
|
||||||
configuration actually achieved.
|
configuration actually achieved. `--calibration-out` fits the timing model
|
||||||
|
across the sweep and writes it as a calibration.
|
||||||
|
- **`calibrate.py`** — converts a raw capture to a nanotesla CSV, applying a
|
||||||
|
calibration when one is named and nothing when one is not. See below.
|
||||||
|
|
||||||
`--trim SECONDS` drops that much from *both* ends of a capture. A run usually
|
`--trim SECONDS` drops that much from *both* ends of a capture. A run usually
|
||||||
opens while the sensor is still settling and closes with a hand on the rig, and
|
opens while the sensor is still settling and closes with a hand on the rig, and
|
||||||
neither end is a noise measurement. Nothing is dropped unless asked, and a
|
neither end is a noise measurement. Nothing is dropped unless asked, and a
|
||||||
capture shorter than 4× the trim is analysed whole with a note rather than gutted.
|
capture shorter than 4× the trim is analysed whole with a note rather than gutted.
|
||||||
|
|
||||||
|
### Calibration
|
||||||
|
|
||||||
|
**Nothing is corrected by default.** `calibrate.py` without `--calibration`
|
||||||
|
changes units and nothing else, and the analysis tools read raw captures as they
|
||||||
|
always did. That is deliberate: the corrections below are real but not yet
|
||||||
|
settled enough to apply silently.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./.venv/bin/python sweep.py --rates 32,64,128,256 \
|
||||||
|
--calibration-out bench.json --note "unit A at 3.006 V"
|
||||||
|
./.venv/bin/python calibrate.py raw.csv -o field.csv --calibration bench.json
|
||||||
|
./.venv/bin/python compare.py --calibration bench.json a.csv b.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
A calibration is **per unit and per supply**, because the chip times everything
|
||||||
|
from one oscillator and that oscillator is neither the specified frequency nor
|
||||||
|
fixed. Solving `period = 3 × (cc/C + overhead)` freely against one unit gave
|
||||||
|
C = 88,546 with a 40.6 µs overhead at 3.006 V, against the nominal 90,000 and
|
||||||
|
68.7 µs — and C moved to 92,889 at 3.353 V.
|
||||||
|
|
||||||
|
That is worth having even if you never correct a field, because it makes the
|
||||||
|
timing model right: with the measured pair, the predicted rate matches the
|
||||||
|
measured one to −0.00% at both cycle counts, where the nominal model is 1.9% out
|
||||||
|
between them. Once it is right, **one capture recovers the oscillator** —
|
||||||
|
`C = cc / (period/3 − overhead)` — to about 0.09% at cycle count 100.
|
||||||
|
|
||||||
|
Two corrections then become available, and neither is on by default:
|
||||||
|
|
||||||
|
| | what it is | status |
|
||||||
|
|---|---|---|
|
||||||
|
| oscillator gain | `(C / reference) ^ gain_exponent` | −7.1% for +0.347 V. The exponent is **1**, which an oscillator-clocked measurement interval predicts and the measured rates support at 0.3σ against 1.2σ for the alternative |
|
||||||
|
| cycle-count gain | `gain_offset_counts` | `gain ∝ (cc + n)`. Table 3-1 implies n = 4.086; measured on one unit it is 0.90 and −0.09. Together the two corrections take a 10.3% disagreement to 2.5%, against a 2.3–3.1% movement floor |
|
||||||
|
|
||||||
|
Every correction arrives with a propagated standard deviation — 0.3–0.4% on
|
||||||
|
these captures, well under the residual, which is what says the residual is real
|
||||||
|
rather than calibration slop. It comes from the timing fit and from
|
||||||
|
`rate_stability()`, which measures how far the oscillator drifted during a run:
|
||||||
|
**186 to 1241 ppm** here, monotonically downward. Let the rig settle first.
|
||||||
|
|
||||||
|
The output is a different format and `capture.py` will refuse to read it, on
|
||||||
|
purpose — raw captures stay the one source of truth. Its header carries the
|
||||||
|
source's SHA-256 so a derived file that has drifted from its source is
|
||||||
|
detectable rather than merely plausible. Full analysis in
|
||||||
|
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
|
||||||
|
|
||||||
|
> `Capture.rate_error` is measured against the **nominal** model, whose error is
|
||||||
|
> cycle-count dependent — +0.84% at cc=100 and −0.99% at cc=400 on one unit, same
|
||||||
|
> supply. It is a configuration diagnostic, not a gain measurement, and
|
||||||
|
> multiplying a field by `1 + rate_error` is not a correction. `calibrate.py`
|
||||||
|
> does that against a measured reference instead.
|
||||||
|
|
||||||
A capture is analysable whether the run ended on `--duration`, on **Ctrl-C**, or
|
A capture is analysable whether the run ended on `--duration`, on **Ctrl-C**, or
|
||||||
early — it is valid up to wherever it stopped, so the analysis tools can be
|
early — it is valid up to wherever it stopped, so the analysis tools can be
|
||||||
pointed at it either way.
|
pointed at it either way.
|
||||||
|
|
@ -528,6 +598,7 @@ than merely "data arrived".
|
||||||
| `characterize.py` | noise floor: spectral density, Allan deviation, residuals, read latency |
|
| `characterize.py` | noise floor: spectral density, Allan deviation, residuals, read latency |
|
||||||
| `sweep.py` | sweep target rates, reporting measured rate, LSB, noise, duty and bus use per point |
|
| `sweep.py` | sweep target rates, reporting measured rate, LSB, noise, duty and bus use per point |
|
||||||
| `compare.py` | A/B captures — fractional noise, plus the checks that separate a gain change from a moved sensor |
|
| `compare.py` | A/B captures — fractional noise, plus the checks that separate a gain change from a moved sensor |
|
||||||
|
| `calibrate.py` | convert a raw capture to nanotesla, applying a per-unit calibration if one is given; also holds the calibration model |
|
||||||
| `diagnose-comms.py` | walks USB → bus → identity → registers → BIST → live read, stopping at the first failure |
|
| `diagnose-comms.py` | walks USB → bus → identity → registers → BIST → live read, stopping at the first failure |
|
||||||
| `tests/` | pytest suite; needs no hardware |
|
| `tests/` | pytest suite; needs no hardware |
|
||||||
| `Noise_Floor_Testing/` | captures, figures and [NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md) — the supply and rate-scaling analysis. Untracked: `.gitignore` excludes it |
|
| `Noise_Floor_Testing/` | captures, figures and [NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md) — the supply and rate-scaling analysis. Untracked: `.gitignore` excludes it |
|
||||||
|
|
@ -636,7 +707,10 @@ inserting anything.
|
||||||
— every pair in the A/B still shows 2.8–23° of rotation
|
— every pair in the A/B still shows 2.8–23° of rotation
|
||||||
- [ ] Identify the **fs/4 artefact** that appears at cc=400 and not at cc=100;
|
- [ ] Identify the **fs/4 artefact** that appears at cc=400 and not at cc=100;
|
||||||
a cc=200 run at both supplies would say whether it scales with cycle count
|
a cc=200 run at both supplies would say whether it scales with cycle count
|
||||||
- [ ] Fix `rm3100.AXIS_OVERHEAD_S`, which is ~1.7× too large for this unit
|
- [x] Per-unit timing and gain calibration — `calibrate.py`, written by
|
||||||
|
`sweep.py --calibration-out`, nothing corrected by default
|
||||||
|
- [ ] Measure a calibration on the bench and check the oscillator against a
|
||||||
|
third rail voltage, which would separate V from V² in the gain scaling
|
||||||
- [ ] Deal with 60 Hz coupling at source
|
- [ ] Deal with 60 Hz coupling at source
|
||||||
- [ ] Reliability testing over extended runs on bare metal
|
- [ ] Reliability testing over extended runs on bare metal
|
||||||
|
|
||||||
|
|
|
||||||
723
calibrate.py
Normal file
723
calibrate.py
Normal file
|
|
@ -0,0 +1,723 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Convert a raw capture to nanotesla, optionally applying a calibration.
|
||||||
|
|
||||||
|
./.venv/bin/python calibrate.py raw.csv -o field.csv
|
||||||
|
./.venv/bin/python calibrate.py raw.csv -o field.csv --calibration bench.json
|
||||||
|
./.venv/bin/python calibrate.py raw.csv -o field.csv --trim 30
|
||||||
|
|
||||||
|
Without `--calibration` this only changes the format: counts become nanotesla
|
||||||
|
using the `tesla_per_count` the capture already carries, and nothing is
|
||||||
|
corrected. That is the default deliberately -- see "What is not corrected".
|
||||||
|
|
||||||
|
**Why a calibration is per-unit.** The chip times everything from one oscillator,
|
||||||
|
and `rm3100`'s model of it, `period = 3 x (cycle_count / C + overhead)`, uses a
|
||||||
|
specified C and an overhead measured on one rig. Neither fits every part. Solving
|
||||||
|
the model against one unit's two cycle counts gave C = 88,546 with a 40.6 us
|
||||||
|
overhead at 3.006 V, against the model's 90,000 and 68.7 us -- and C moved to
|
||||||
|
92,889 when the supply moved to 3.353 V, because the oscillator does.
|
||||||
|
|
||||||
|
That matters beyond timing. Gain moves with the oscillator, so a capture taken at
|
||||||
|
a different supply reads a different field for the same magnet. A calibration
|
||||||
|
records the timing pair and a reference oscillator frequency, and the correction
|
||||||
|
is how far this capture's oscillator sits from that reference.
|
||||||
|
|
||||||
|
Once the timing pair is right, **one capture is enough** to recover the
|
||||||
|
oscillator: rearranging the model gives `C = cycle_count / (period/3 - overhead)`,
|
||||||
|
and a 1 us error in the overhead moves C by only 0.09% at cycle count 100. With
|
||||||
|
the model's 28 us error it moves by 2.5%, and by a *different* amount at each
|
||||||
|
cycle count, which is what makes the uncalibrated figures unusable for this.
|
||||||
|
|
||||||
|
**Why the exponent is 1.** The measurement interval is clocked entirely by that
|
||||||
|
oscillator: the per-axis overhead measures 3.60 counts at 3.006 V and 3.54 at
|
||||||
|
3.353 V while its *duration* moves 6%, so the same number of counts is always
|
||||||
|
collected over a window whose length goes as 1/f. Gain then follows integration
|
||||||
|
time, giving `gain ~ (cc + n) / f` -- exponent exactly 1. Measured rates agree:
|
||||||
|
the supply rate ratio is the same at both cycle counts to 0.04%, which a
|
||||||
|
fixed-time overhead would not produce. It is favoured rather than proven, at 0.3
|
||||||
|
sigma against 1.2 for a fixed-time overhead; thermal drift is what limits it.
|
||||||
|
|
||||||
|
**Why there is a gain offset.** Table 3-1's `0.3671*cc + 1.5` LSB/uT implies the
|
||||||
|
gain goes as `cc + 4.086` in count units. Solving the same-supply cycle-count
|
||||||
|
contrast on one unit gives 0.90 and -0.09 -- near zero, so the overhead counts
|
||||||
|
cost time without integrating field. Leaving it at the datasheet value accounts
|
||||||
|
for only half of what the correction can remove.
|
||||||
|
|
||||||
|
**What is not corrected, and why nothing is by default.** All of it was measured
|
||||||
|
on four captures, two per supply, with a sensor that rotated between runs by 2.8
|
||||||
|
to 23 degrees, and every capture drifts 186 to 1241 ppm through its own thermal
|
||||||
|
warm-up. So it is applied only when a calibration is named, every factor arrives
|
||||||
|
with the standard deviation it earned, and the output records what was done.
|
||||||
|
|
||||||
|
The output is a different format from a raw capture and `capture.py` will refuse
|
||||||
|
to read it, which is intended: raw captures stay the one source of truth, and a
|
||||||
|
derived file that drifts from its source should be detectable rather than
|
||||||
|
plausible. The header carries the source's SHA-256 for that reason.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import csv
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from collections import namedtuple
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
import capture
|
||||||
|
import characterize as ch
|
||||||
|
import rm3100
|
||||||
|
|
||||||
|
# Bumped when either file format changes incompatibly, so a reader can refuse
|
||||||
|
# rather than misinterpret.
|
||||||
|
CALIBRATION_VERSION = 2
|
||||||
|
CALIBRATED_VERSION = 1
|
||||||
|
|
||||||
|
# Read in blocks when hashing: a capture can be tens of megabytes and there is
|
||||||
|
# no reason to hold one in memory twice.
|
||||||
|
HASH_BLOCK = 1 << 20
|
||||||
|
|
||||||
|
# Sub-windows used to measure how much the oscillator moved during a capture.
|
||||||
|
# Eight over a five-minute run is ~40 s each, long enough to fit a rate against
|
||||||
|
# and short enough to see a thermal transient.
|
||||||
|
STABILITY_WINDOWS = 8
|
||||||
|
|
||||||
|
|
||||||
|
class CalibrationError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class Estimate(namedtuple("Estimate", "value sd")):
|
||||||
|
"""A number and its standard deviation, carried together.
|
||||||
|
|
||||||
|
Every corrected field here rests on a chain of measurements, and a residual
|
||||||
|
only means something against the uncertainty of the thing that was supposed
|
||||||
|
to remove it. Returning the two together is what makes that comparison
|
||||||
|
possible at the point of use rather than in a comment.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__slots__ = ()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def relative(self):
|
||||||
|
return self.sd / abs(self.value) if self.value else float("inf")
|
||||||
|
|
||||||
|
def __format__(self, spec):
|
||||||
|
return f"{self.value:{spec or '.6g'}} +/- {self.sd:{spec or '.2g'}}"
|
||||||
|
|
||||||
|
|
||||||
|
Calibration = namedtuple("Calibration", [
|
||||||
|
"counts_per_second", # C in the timing model, this unit's own
|
||||||
|
"axis_overhead_s", # the model's fixed per-axis term
|
||||||
|
"reference_oscillator_hz", # C at which the gain is taken as correct
|
||||||
|
"reference_cycle_count", # cycle count at which it is taken as correct
|
||||||
|
"gain_exponent", # gain ~ (cc + offset) / f ** exponent
|
||||||
|
"gain_offset_counts", # the "+ n" -- see gain_factor()
|
||||||
|
"counts_per_second_sd",
|
||||||
|
"axis_overhead_s_sd",
|
||||||
|
"gain_exponent_sd",
|
||||||
|
"gain_offset_counts_sd",
|
||||||
|
"note",
|
||||||
|
"created",
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
|
def calibration(counts_per_second, axis_overhead_s,
|
||||||
|
reference_oscillator_hz=None, reference_cycle_count=100,
|
||||||
|
gain_exponent=1.0, gain_offset_counts=0.0,
|
||||||
|
counts_per_second_sd=0.0, axis_overhead_s_sd=0.0,
|
||||||
|
gain_exponent_sd=0.0, gain_offset_counts_sd=0.0,
|
||||||
|
note="", created=""):
|
||||||
|
"""Build a Calibration, defaulting the reference to the measured value.
|
||||||
|
|
||||||
|
A calibration whose reference is its own oscillator frequency corrects
|
||||||
|
nothing on the unit and supply it was taken at, and corrects other captures
|
||||||
|
relative to it. That is the useful default: the reference is a choice of
|
||||||
|
where "right" is, and the only defensible choice without an absolute field
|
||||||
|
standard is the condition the gain was characterised at.
|
||||||
|
|
||||||
|
The uncertainties all default to zero, so a calibration that makes no claim
|
||||||
|
about its own accuracy still works -- and every field it corrects then
|
||||||
|
reports a standard deviation of zero, which is a visible claim rather than a
|
||||||
|
silent one.
|
||||||
|
"""
|
||||||
|
return Calibration(
|
||||||
|
counts_per_second=float(counts_per_second),
|
||||||
|
axis_overhead_s=float(axis_overhead_s),
|
||||||
|
reference_oscillator_hz=float(reference_oscillator_hz
|
||||||
|
if reference_oscillator_hz is not None
|
||||||
|
else counts_per_second),
|
||||||
|
reference_cycle_count=int(reference_cycle_count),
|
||||||
|
gain_exponent=float(gain_exponent),
|
||||||
|
gain_offset_counts=float(gain_offset_counts),
|
||||||
|
counts_per_second_sd=float(counts_per_second_sd),
|
||||||
|
axis_overhead_s_sd=float(axis_overhead_s_sd),
|
||||||
|
gain_exponent_sd=float(gain_exponent_sd),
|
||||||
|
gain_offset_counts_sd=float(gain_offset_counts_sd),
|
||||||
|
note=str(note), created=str(created),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _finite(value, field):
|
||||||
|
"""A field that must parse as a real number. Booleans are not numbers here.
|
||||||
|
|
||||||
|
JSON's `true` reaches Python as a bool, which floats to 1.0 without
|
||||||
|
complaint -- a silent way for a malformed calibration to become a plausible
|
||||||
|
one.
|
||||||
|
"""
|
||||||
|
if isinstance(value, bool) or not isinstance(value, (int, float, str)):
|
||||||
|
raise CalibrationError(f"{field}: {value!r} is not a number")
|
||||||
|
try:
|
||||||
|
number = float(value)
|
||||||
|
except ValueError:
|
||||||
|
raise CalibrationError(f"{field}: {value!r} is not a number")
|
||||||
|
if not math.isfinite(number):
|
||||||
|
raise CalibrationError(f"{field}: {value!r} is not finite")
|
||||||
|
return number
|
||||||
|
|
||||||
|
|
||||||
|
def _positive(value, field):
|
||||||
|
number = _finite(value, field)
|
||||||
|
if number <= 0:
|
||||||
|
raise CalibrationError(f"{field}: {number} must be positive")
|
||||||
|
return number
|
||||||
|
|
||||||
|
|
||||||
|
def _non_negative(value, field):
|
||||||
|
number = _finite(value, field)
|
||||||
|
if number < 0:
|
||||||
|
raise CalibrationError(f"{field}: {number} must not be negative")
|
||||||
|
return number
|
||||||
|
|
||||||
|
|
||||||
|
def load_calibration(path):
|
||||||
|
"""Read a calibration file, refusing anything it cannot fully understand."""
|
||||||
|
try:
|
||||||
|
with open(path) as handle:
|
||||||
|
raw = json.load(handle)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise CalibrationError(f"{path}: not valid JSON ({exc})")
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
raise CalibrationError(f"{path}: expected an object, got "
|
||||||
|
f"{type(raw).__name__}")
|
||||||
|
version = raw.get("rm3100_calibration")
|
||||||
|
if version != CALIBRATION_VERSION:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{path}: rm3100_calibration is {version!r}, this reads "
|
||||||
|
f"{CALIBRATION_VERSION}. Version 1 carried a per-cycle-count gain "
|
||||||
|
"table; version 2 replaces it with gain_offset_counts, which is "
|
||||||
|
"not derivable from it -- re-measure rather than convert.")
|
||||||
|
missing = {"counts_per_second", "axis_overhead_s"} - raw.keys()
|
||||||
|
if missing:
|
||||||
|
raise CalibrationError(f"{path}: missing {', '.join(sorted(missing))}")
|
||||||
|
|
||||||
|
cycle_count = raw.get("reference_cycle_count", 100)
|
||||||
|
if isinstance(cycle_count, bool) or not isinstance(cycle_count, (int, str)):
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{path}: reference_cycle_count {cycle_count!r} is not an integer")
|
||||||
|
try:
|
||||||
|
cycle_count = int(cycle_count)
|
||||||
|
except ValueError:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{path}: reference_cycle_count {cycle_count!r} is not an integer")
|
||||||
|
if cycle_count < 1:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{path}: reference_cycle_count {cycle_count} must be at least 1")
|
||||||
|
|
||||||
|
return Calibration(
|
||||||
|
counts_per_second=_positive(raw["counts_per_second"],
|
||||||
|
"counts_per_second"),
|
||||||
|
# The overhead is a duration and may legitimately be zero, but not
|
||||||
|
# negative: a measurement cannot take less time than its own counts.
|
||||||
|
axis_overhead_s=_non_negative(raw["axis_overhead_s"],
|
||||||
|
"axis_overhead_s"),
|
||||||
|
reference_oscillator_hz=_positive(
|
||||||
|
raw.get("reference_oscillator_hz", raw["counts_per_second"]),
|
||||||
|
"reference_oscillator_hz"),
|
||||||
|
reference_cycle_count=cycle_count,
|
||||||
|
gain_exponent=_finite(raw.get("gain_exponent", 1.0), "gain_exponent"),
|
||||||
|
gain_offset_counts=_finite(raw.get("gain_offset_counts", 0.0),
|
||||||
|
"gain_offset_counts"),
|
||||||
|
counts_per_second_sd=_non_negative(
|
||||||
|
raw.get("counts_per_second_sd", 0.0), "counts_per_second_sd"),
|
||||||
|
axis_overhead_s_sd=_non_negative(
|
||||||
|
raw.get("axis_overhead_s_sd", 0.0), "axis_overhead_s_sd"),
|
||||||
|
gain_exponent_sd=_non_negative(
|
||||||
|
raw.get("gain_exponent_sd", 0.0), "gain_exponent_sd"),
|
||||||
|
gain_offset_counts_sd=_non_negative(
|
||||||
|
raw.get("gain_offset_counts_sd", 0.0), "gain_offset_counts_sd"),
|
||||||
|
note=str(raw.get("note", "")),
|
||||||
|
created=str(raw.get("created", "")),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def save_calibration(path, cal):
|
||||||
|
"""Write a calibration file."""
|
||||||
|
if cal.axis_overhead_s < 0:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"axis_overhead_s {cal.axis_overhead_s} is negative")
|
||||||
|
body = {"rm3100_calibration": CALIBRATION_VERSION,
|
||||||
|
"note": cal.note, "created": cal.created}
|
||||||
|
body.update({field: getattr(cal, field) for field in Calibration._fields
|
||||||
|
if field not in ("note", "created")})
|
||||||
|
with open(path, "w") as handle:
|
||||||
|
json.dump(body, handle, indent=2, sort_keys=False)
|
||||||
|
handle.write("\n")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def window_rates(cap, seconds):
|
||||||
|
"""Sample rate fitted independently on each consecutive window of a capture.
|
||||||
|
|
||||||
|
Returns (centres, rates) -- seconds from the start of the capture, and Hz --
|
||||||
|
with one point per window that held enough samples to fit.
|
||||||
|
|
||||||
|
`Capture.dt_true` is a single straight line through a run whose rate is
|
||||||
|
moving. This is the same fit repeated on pieces short enough that the
|
||||||
|
movement within any one of them is small, which is what turns the drift from
|
||||||
|
an error on the whole-capture number into something measurable in its own
|
||||||
|
right.
|
||||||
|
"""
|
||||||
|
seconds = float(seconds)
|
||||||
|
start = float(cap.elapsed[0])
|
||||||
|
if seconds <= 0 or cap.duration < 2 * seconds:
|
||||||
|
return np.array([]), np.array([])
|
||||||
|
centres, rates = [], []
|
||||||
|
for w in range(int(cap.duration // seconds)):
|
||||||
|
low = start + w * seconds
|
||||||
|
try:
|
||||||
|
rate = cap.restrict(low, low + seconds).true_rate_hz
|
||||||
|
except capture.CaptureError:
|
||||||
|
continue
|
||||||
|
rates.append(rate)
|
||||||
|
centres.append((w + 0.5) * seconds)
|
||||||
|
return np.array(centres), np.array(rates)
|
||||||
|
|
||||||
|
|
||||||
|
def rate_stability(cap, windows=STABILITY_WINDOWS):
|
||||||
|
"""Fractional spread of the sample rate across sub-windows of a capture.
|
||||||
|
|
||||||
|
The chip's RC oscillator warms up, and every capture here drifts downward
|
||||||
|
through its own run -- 186 ppm on the quietest, 1241 ppm on the worst. A
|
||||||
|
single fitted `dt_true` is the average over that transient, so this is the
|
||||||
|
honest error bar on it, and the input to every uncertainty downstream.
|
||||||
|
|
||||||
|
It is not the same thing as `Capture.drift_limited`, which asks whether one
|
||||||
|
straight line describes the capture at all. This asks how far the rate moved
|
||||||
|
while it did. Nor is it `rate_allan`, which asks how much of that movement
|
||||||
|
is real: a spread this wide is only meaningful against the noise of the
|
||||||
|
measurement that produced it.
|
||||||
|
"""
|
||||||
|
if len(cap.sample_index) < 64 * windows:
|
||||||
|
return 0.0
|
||||||
|
_, rates = window_rates(cap, cap.duration / windows)
|
||||||
|
if len(rates) < 2:
|
||||||
|
return 0.0
|
||||||
|
return float((rates.max() - rates.min()) / rates.mean())
|
||||||
|
|
||||||
|
|
||||||
|
def rate_allan(cap, taus):
|
||||||
|
"""Two-sample (Allan) deviation of the measured sample rate, fractional.
|
||||||
|
|
||||||
|
For each averaging time tau the rate is fitted on consecutive tau-second
|
||||||
|
windows and the deviation taken between neighbours:
|
||||||
|
|
||||||
|
sigma_y(tau) = sqrt( mean( (y[k+1] - y[k])^2 ) / 2 )
|
||||||
|
|
||||||
|
Differences between neighbours rather than a spread about the mean, for the
|
||||||
|
same reason `characterize.white_sd` differences rather than taking a
|
||||||
|
standard deviation: on a series that drifts, the spread measures the drift.
|
||||||
|
Here both effects are wanted and the shape of the curve separates them.
|
||||||
|
Falling with tau is measurement noise averaging down; rising with tau is
|
||||||
|
drift the measurement has begun to resolve, and a rise proportional to tau
|
||||||
|
specifically is a deterministic frequency ramp rather than a random walk.
|
||||||
|
|
||||||
|
The minimum is therefore the best the rate can be known, and the tau where
|
||||||
|
it falls is how long it is worth measuring for.
|
||||||
|
|
||||||
|
Returns (taus, sigma_y) over the taus that yielded three or more windows.
|
||||||
|
Two windows give a single difference, which is not an estimate of anything.
|
||||||
|
"""
|
||||||
|
kept, devs = [], []
|
||||||
|
for tau in taus:
|
||||||
|
_, rates = window_rates(cap, tau)
|
||||||
|
if len(rates) < 3:
|
||||||
|
continue
|
||||||
|
y = rates / rates.mean()
|
||||||
|
kept.append(float(tau))
|
||||||
|
devs.append(float(np.sqrt(np.mean(np.diff(y) ** 2) / 2)))
|
||||||
|
return np.array(kept), np.array(devs)
|
||||||
|
|
||||||
|
|
||||||
|
TimingFit = namedtuple("TimingFit", "counts_per_second axis_overhead_s "
|
||||||
|
"counts_per_second_sd axis_overhead_s_sd "
|
||||||
|
"residual_ppm points")
|
||||||
|
|
||||||
|
|
||||||
|
def fit_timing(cycle_counts, periods, period_sd=None):
|
||||||
|
"""Solve `period = AXES * (cycle_count / C + overhead)` for both terms.
|
||||||
|
|
||||||
|
`periods` are whole sample periods in seconds, not per-axis. `period_sd` is
|
||||||
|
the fractional uncertainty on each -- `rate_stability()` is where it comes
|
||||||
|
from.
|
||||||
|
|
||||||
|
Both terms are needed. Holding C at the specified 90,000 and solving for the
|
||||||
|
overhead alone gave 58.9 us at cycle count 100 and 113.6 us at 400 on the
|
||||||
|
same unit -- no single value fits, because the misfit is in both.
|
||||||
|
|
||||||
|
**Two points determine both terms exactly and leave no residual**, so a bad
|
||||||
|
point cannot be detected: `residual_ppm` is 0 and means nothing. That case
|
||||||
|
requires `period_sd`, so the uncertainty is at least stated rather than
|
||||||
|
fabricated from a perfect fit. Three or more points carry their own.
|
||||||
|
"""
|
||||||
|
counts = np.asarray(cycle_counts, dtype=float)
|
||||||
|
per_axis = np.asarray(periods, dtype=float) / rm3100.AXES
|
||||||
|
if counts.shape != per_axis.shape:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{counts.size} cycle counts against {per_axis.size} periods")
|
||||||
|
distinct = len(np.unique(counts))
|
||||||
|
if distinct < 2:
|
||||||
|
raise CalibrationError(
|
||||||
|
"need at least two distinct cycle counts to fit both the count "
|
||||||
|
"rate and the per-axis overhead")
|
||||||
|
|
||||||
|
if distinct == 2 and len(counts) == 2:
|
||||||
|
if period_sd is None:
|
||||||
|
raise CalibrationError(
|
||||||
|
"a two-point fit has no residual and cannot detect a bad "
|
||||||
|
"point, so it cannot estimate its own uncertainty. Pass "
|
||||||
|
"period_sd (from rate_stability) or sweep a third cycle count.")
|
||||||
|
slope, intercept = np.polyfit(counts, per_axis, 1)
|
||||||
|
if slope <= 0:
|
||||||
|
raise CalibrationError(
|
||||||
|
"fitted count rate is not positive; these captures do not "
|
||||||
|
"follow the timing model")
|
||||||
|
count_rate = 1.0 / float(slope)
|
||||||
|
# C leans on the *difference* of the two per-axis times, so a fractional
|
||||||
|
# error on each is amplified by how close together they are.
|
||||||
|
span = abs(per_axis[1] - per_axis[0])
|
||||||
|
rate_sd = count_rate * period_sd * float(np.hypot(*per_axis)) / span
|
||||||
|
overhead_sd = (per_axis.min() * period_sd
|
||||||
|
+ counts.min() / count_rate * rate_sd / count_rate)
|
||||||
|
return TimingFit(count_rate, float(intercept), rate_sd, overhead_sd,
|
||||||
|
0.0, len(counts))
|
||||||
|
|
||||||
|
(slope, intercept), cov = np.polyfit(counts, per_axis, 1, cov=True)
|
||||||
|
if slope <= 0:
|
||||||
|
raise CalibrationError(
|
||||||
|
"fitted count rate is not positive; these captures do not follow "
|
||||||
|
"the timing model")
|
||||||
|
count_rate = 1.0 / float(slope)
|
||||||
|
slope_sd, overhead_sd = math.sqrt(cov[0, 0]), math.sqrt(cov[1, 1])
|
||||||
|
residual = per_axis - (slope * counts + intercept)
|
||||||
|
return TimingFit(
|
||||||
|
count_rate, float(intercept),
|
||||||
|
# d(1/slope) = dslope / slope^2
|
||||||
|
float(slope_sd) / slope ** 2, float(overhead_sd),
|
||||||
|
float(np.abs(residual / per_axis).max() * 1e6), len(counts))
|
||||||
|
|
||||||
|
|
||||||
|
def oscillator_hz(cap, cal, period_sd=None):
|
||||||
|
"""This capture's oscillator frequency, with its uncertainty.
|
||||||
|
|
||||||
|
Inverts `period = AXES * (cycle_count / C + overhead)`. The overhead comes
|
||||||
|
from the calibration because it cannot be recovered from a single cycle
|
||||||
|
count -- one equation, two unknowns -- and it is the term the nominal model
|
||||||
|
gets most wrong.
|
||||||
|
|
||||||
|
`period_sd` is the fractional uncertainty on the measured period; it
|
||||||
|
defaults to measuring it from the capture with `rate_stability()`.
|
||||||
|
"""
|
||||||
|
per_axis = cap.dt_true / rm3100.AXES - cal.axis_overhead_s
|
||||||
|
if per_axis <= 0:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{cap.path}: a {cap.dt_true * 1e3:.3f} ms period leaves no time "
|
||||||
|
f"for {cap.cycle_count} counts once the calibration's "
|
||||||
|
f"{cal.axis_overhead_s * 1e6:.1f} us overhead is removed. The "
|
||||||
|
"calibration does not describe this capture.")
|
||||||
|
if period_sd is None:
|
||||||
|
period_sd = rate_stability(cap)
|
||||||
|
value = cap.cycle_count / per_axis
|
||||||
|
# Two independent contributions: how well the period was measured, and how
|
||||||
|
# well the overhead subtracted from it is known. The second dominates at low
|
||||||
|
# cycle counts, where the overhead is a larger share of the per-axis time.
|
||||||
|
from_period = cap.dt_true * period_sd / rm3100.AXES
|
||||||
|
relative = math.hypot(from_period, cal.axis_overhead_s_sd) / per_axis
|
||||||
|
return Estimate(value, value * relative)
|
||||||
|
|
||||||
|
|
||||||
|
def gain_factor(cap, cal, period_sd=None):
|
||||||
|
"""Multiply measured nanotesla by this to put them on the reference gain.
|
||||||
|
|
||||||
|
Three terms, each correcting something the capture's own conversion assumed:
|
||||||
|
|
||||||
|
oscillator how far this capture's clock sits from the reference, raised
|
||||||
|
to the fitted exponent. The measurement interval is clocked
|
||||||
|
entirely by that oscillator -- the per-axis overhead measures
|
||||||
|
3.57 counts at both supplies, constant, while its *duration*
|
||||||
|
moves 6% -- so the same number of counts is always collected
|
||||||
|
over a window whose length goes as 1/f. That predicts an
|
||||||
|
exponent of exactly 1.
|
||||||
|
gain shape Table 3-1's `0.3671*cc + 1.5` LSB/uT implies the gain goes as
|
||||||
|
`cc + 4.086` in count units. Measured on one unit it is 0 to
|
||||||
|
0.9, so the overhead counts cost time without integrating
|
||||||
|
field. `gain_offset_counts` is that "+ n".
|
||||||
|
reference the shape term is only known up to an overall scale, since
|
||||||
|
`gain = A(cc + n)` leaves A free. `reference_cycle_count`
|
||||||
|
pins it: the correction is exactly 1 there.
|
||||||
|
"""
|
||||||
|
oscillator = oscillator_hz(cap, cal, period_sd)
|
||||||
|
ratio = oscillator.value / cal.reference_oscillator_hz
|
||||||
|
reference_cc, offset = cal.reference_cycle_count, cal.gain_offset_counts
|
||||||
|
if cap.cycle_count + offset <= 0 or reference_cc + offset <= 0:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"gain_offset_counts {offset} puts the gain at or below zero for "
|
||||||
|
f"cycle count {min(cap.cycle_count, reference_cc)}")
|
||||||
|
shape = ((rm3100.gain_lsb_per_tesla(cap.cycle_count)
|
||||||
|
/ rm3100.gain_lsb_per_tesla(reference_cc))
|
||||||
|
* ((reference_cc + offset) / (cap.cycle_count + offset)))
|
||||||
|
value = ratio ** cal.gain_exponent * shape
|
||||||
|
|
||||||
|
# In logs, so the terms add in quadrature as independent contributions.
|
||||||
|
from_oscillator = cal.gain_exponent * oscillator.relative
|
||||||
|
from_exponent = cal.gain_exponent_sd * abs(math.log(ratio)) if ratio > 0 else 0.0
|
||||||
|
from_offset = cal.gain_offset_counts_sd * abs(
|
||||||
|
1.0 / (reference_cc + offset) - 1.0 / (cap.cycle_count + offset))
|
||||||
|
return Estimate(value, value * math.sqrt(from_oscillator ** 2
|
||||||
|
+ from_exponent ** 2
|
||||||
|
+ from_offset ** 2))
|
||||||
|
|
||||||
|
|
||||||
|
def source_digest(path):
|
||||||
|
"""SHA-256 of a file, so a derived copy can be tied back to its source."""
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
with open(path, "rb") as handle:
|
||||||
|
for block in iter(lambda: handle.read(HASH_BLOCK), b""):
|
||||||
|
digest.update(block)
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def write_calibrated(cap, path, source, cal=None, factor=None,
|
||||||
|
calibration_path=None):
|
||||||
|
"""Write nanotesla and a header recording where they came from.
|
||||||
|
|
||||||
|
`factor` is an Estimate; its standard deviation goes into the header beside
|
||||||
|
it, so a reader can tell a real residual from calibration slop without
|
||||||
|
having the calibration to hand.
|
||||||
|
"""
|
||||||
|
if factor is None:
|
||||||
|
factor = Estimate(1.0, 0.0)
|
||||||
|
elif not isinstance(factor, Estimate):
|
||||||
|
factor = Estimate(float(factor), 0.0)
|
||||||
|
header = [
|
||||||
|
("rm3100_calibrated", CALIBRATED_VERSION),
|
||||||
|
("source", os.path.basename(source)),
|
||||||
|
("source_sha256", source_digest(source)),
|
||||||
|
("samples", len(cap.sample_index)),
|
||||||
|
("cycle_count", cap.cycle_count),
|
||||||
|
("tesla_per_count", repr(cap.tesla_per_count)),
|
||||||
|
("measured_rate_hz", repr(cap.true_rate_hz)),
|
||||||
|
("calibration", os.path.basename(calibration_path)
|
||||||
|
if calibration_path else "none"),
|
||||||
|
("gain_factor", repr(factor.value)),
|
||||||
|
("gain_factor_sd", repr(factor.sd)),
|
||||||
|
]
|
||||||
|
if cal is not None:
|
||||||
|
oscillator = oscillator_hz(cap, cal)
|
||||||
|
header += [
|
||||||
|
("oscillator_hz", repr(oscillator.value)),
|
||||||
|
("oscillator_hz_sd", repr(oscillator.sd)),
|
||||||
|
("reference_oscillator_hz", repr(cal.reference_oscillator_hz)),
|
||||||
|
("reference_cycle_count", cal.reference_cycle_count),
|
||||||
|
("gain_exponent", repr(cal.gain_exponent)),
|
||||||
|
("gain_offset_counts", repr(cal.gain_offset_counts)),
|
||||||
|
]
|
||||||
|
|
||||||
|
x, y, z = (cap.x * factor.value, cap.y * factor.value,
|
||||||
|
cap.z * factor.value)
|
||||||
|
total = np.sqrt(x**2 + y**2 + z**2)
|
||||||
|
# The two flags are independent, so a row can carry both. Placeholder rows
|
||||||
|
# keep their interpolated values here, exactly as capture.py produced them,
|
||||||
|
# and the flag is what says the values were reconstructed.
|
||||||
|
flags = [" ".join(
|
||||||
|
([capture.WARN_MISSED] if missed else [])
|
||||||
|
+ ([capture.WARN_AMBIGUOUS] if ambiguous else []))
|
||||||
|
for missed, ambiguous in zip(cap.missed, cap.ambiguous)]
|
||||||
|
|
||||||
|
with open(path, "w", newline="") as handle:
|
||||||
|
for key, value in header:
|
||||||
|
handle.write(f"# {key}: {value}\n")
|
||||||
|
writer = csv.writer(handle)
|
||||||
|
writer.writerow(["sample_index", "elapsed_s",
|
||||||
|
"x_nT", "y_nT", "z_nT", "total_nT", "warning"])
|
||||||
|
writer.writerows(
|
||||||
|
(index, f"{t:.6f}", f"{a:.3f}", f"{b:.3f}", f"{c:.3f}",
|
||||||
|
f"{n:.3f}", flag)
|
||||||
|
for index, t, a, b, c, n, flag
|
||||||
|
in zip(cap.sample_index, cap.elapsed, x, y, z, total, flags))
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
class Calibrated:
|
||||||
|
"""A capture already converted to nanotesla, read back from disk.
|
||||||
|
|
||||||
|
Deliberately not a `capture.Capture`: the raw counts and the host clock are
|
||||||
|
gone, so the time base cannot be re-fitted and `restrict()` would be a lie.
|
||||||
|
What survives is what a plot or a spectrum needs, plus the provenance of how
|
||||||
|
it got that way.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, path, meta, sample_index, elapsed, x, y, z, total):
|
||||||
|
self.path = path
|
||||||
|
self.meta = meta
|
||||||
|
self.sample_index = sample_index
|
||||||
|
self.elapsed = elapsed
|
||||||
|
self.x, self.y, self.z, self.total = x, y, z, total
|
||||||
|
self.cycle_count = int(meta["cycle_count"])
|
||||||
|
self.tesla_per_count = float(meta["tesla_per_count"])
|
||||||
|
self.lsb_nt = self.tesla_per_count * rm3100.NT_PER_TESLA
|
||||||
|
self.true_rate_hz = float(meta["measured_rate_hz"])
|
||||||
|
self.dt_true = 1.0 / self.true_rate_hz
|
||||||
|
self.gain_factor = float(meta["gain_factor"])
|
||||||
|
self.gain_factor_sd = float(meta.get("gain_factor_sd", 0.0))
|
||||||
|
self.calibration_name = meta.get("calibration", "none")
|
||||||
|
self.source = meta.get("source", "")
|
||||||
|
self.source_sha256 = meta.get("source_sha256", "")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def duration(self):
|
||||||
|
return float(self.elapsed[-1] - self.elapsed[0])
|
||||||
|
|
||||||
|
def axes(self):
|
||||||
|
return [("x", self.x), ("y", self.y), ("z", self.z),
|
||||||
|
("total", self.total)]
|
||||||
|
|
||||||
|
def describe(self):
|
||||||
|
if self.gain_factor == 1.0:
|
||||||
|
return "converted to nT, uncorrected"
|
||||||
|
return (f"gain x{self.gain_factor:.6f} +/- {self.gain_factor_sd:.6f} "
|
||||||
|
f"from {self.calibration_name}")
|
||||||
|
|
||||||
|
|
||||||
|
def load_calibrated(path):
|
||||||
|
"""Read a file written by write_calibrated()."""
|
||||||
|
meta, rows = {}, []
|
||||||
|
with open(path, newline="") as handle:
|
||||||
|
while True:
|
||||||
|
position = handle.tell()
|
||||||
|
line = handle.readline()
|
||||||
|
if not line:
|
||||||
|
break
|
||||||
|
if not line.startswith("#"):
|
||||||
|
handle.seek(position)
|
||||||
|
break
|
||||||
|
key, _, value = line[1:].partition(":")
|
||||||
|
if value:
|
||||||
|
meta[key.strip()] = value.strip()
|
||||||
|
rows = list(csv.DictReader(handle))
|
||||||
|
|
||||||
|
version = meta.get("rm3100_calibrated")
|
||||||
|
if version is None:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{path}: no rm3100_calibrated header. A raw capture goes through "
|
||||||
|
"capture.load() instead.")
|
||||||
|
if int(version) != CALIBRATED_VERSION:
|
||||||
|
raise CalibrationError(
|
||||||
|
f"{path}: rm3100_calibrated is {version}, this reads "
|
||||||
|
f"{CALIBRATED_VERSION}")
|
||||||
|
missing = {"cycle_count", "tesla_per_count", "measured_rate_hz",
|
||||||
|
"gain_factor"} - meta.keys()
|
||||||
|
if missing:
|
||||||
|
raise CalibrationError(f"{path}: header missing "
|
||||||
|
f"{', '.join(sorted(missing))}")
|
||||||
|
if not rows:
|
||||||
|
raise CalibrationError(f"{path}: no samples")
|
||||||
|
|
||||||
|
column = lambda name: np.array([float(r[name]) for r in rows])
|
||||||
|
return Calibrated(
|
||||||
|
path, meta,
|
||||||
|
np.array([int(r["sample_index"]) for r in rows], dtype=np.int64),
|
||||||
|
column("elapsed_s"), column("x_nT"), column("y_nT"), column("z_nT"),
|
||||||
|
column("total_nT"))
|
||||||
|
|
||||||
|
|
||||||
|
def load_any(path):
|
||||||
|
"""Load a raw capture or a calibrated one, whichever this is.
|
||||||
|
|
||||||
|
Which format a file is, is a property of the file, not of the caller -- so
|
||||||
|
tools take a path and get back something with axes() and a time base either
|
||||||
|
way, rather than each having to know.
|
||||||
|
"""
|
||||||
|
with open(path) as handle:
|
||||||
|
head = handle.read(4096)
|
||||||
|
if "rm3100_calibrated" in head:
|
||||||
|
return load_calibrated(path)
|
||||||
|
return capture.load(path)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(
|
||||||
|
description=__doc__,
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
ap.add_argument("csv", help="raw capture written by logger.py")
|
||||||
|
ap.add_argument("-o", "--output", required=True, metavar="CSV",
|
||||||
|
help="where to write the nanotesla capture")
|
||||||
|
ap.add_argument("--calibration", default=None, metavar="JSON",
|
||||||
|
help="apply this calibration; without it nothing is "
|
||||||
|
"corrected and only the units change")
|
||||||
|
ap.add_argument("--trim", type=float, default=0.0, metavar="SECONDS",
|
||||||
|
help="drop this many seconds from BOTH ends before "
|
||||||
|
"converting (default: %(default)s)")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
# Check the destination before reading a capture that may be tens of
|
||||||
|
# megabytes, so a mistyped directory fails in a second rather than a minute.
|
||||||
|
parent = os.path.dirname(os.path.abspath(args.output)) or "."
|
||||||
|
if not os.path.isdir(parent):
|
||||||
|
sys.exit(f"{args.output}: {parent} is not a directory")
|
||||||
|
if not os.access(parent, os.W_OK):
|
||||||
|
sys.exit(f"{args.output}: {parent} is not writable")
|
||||||
|
|
||||||
|
try:
|
||||||
|
cal = load_calibration(args.calibration) if args.calibration else None
|
||||||
|
cap = capture.load(args.csv)
|
||||||
|
cap, note = ch.trimmed(cap, args.trim)
|
||||||
|
factor = gain_factor(cap, cal) if cal else Estimate(1.0, 0.0)
|
||||||
|
except (OSError, ValueError, CalibrationError,
|
||||||
|
capture.CaptureError) as exc:
|
||||||
|
sys.exit(str(exc))
|
||||||
|
|
||||||
|
if note:
|
||||||
|
print(f" {note}")
|
||||||
|
if cal is None:
|
||||||
|
print("no calibration given: converting units only, correcting nothing")
|
||||||
|
else:
|
||||||
|
measured = oscillator_hz(cap, cal)
|
||||||
|
stability = rate_stability(cap)
|
||||||
|
print(f"calibration {args.calibration}"
|
||||||
|
+ (f" -- {cal.note}" if cal.note else ""))
|
||||||
|
print(f" rate {cap.true_rate_hz:.4f} Hz, drifting "
|
||||||
|
f"{stability * 1e6:.0f} ppm across the run")
|
||||||
|
print(f" oscillator {measured.value:,.0f} +/- {measured.sd:,.0f} Hz "
|
||||||
|
f"against a {cal.reference_oscillator_hz:,.0f} Hz reference "
|
||||||
|
f"({measured.value / cal.reference_oscillator_hz - 1:+.2%})")
|
||||||
|
if cap.cycle_count != cal.reference_cycle_count:
|
||||||
|
print(f" gain shape cycle count {cap.cycle_count} against a "
|
||||||
|
f"{cal.reference_cycle_count} reference, offset "
|
||||||
|
f"{cal.gain_offset_counts:+.2f} counts")
|
||||||
|
print(f" gain factor {factor.value:.6f} +/- {factor.sd:.6f} "
|
||||||
|
f"({factor.relative:.2%}) |B| {cap.total.mean():,.0f} -> "
|
||||||
|
f"{cap.total.mean() * factor.value:,.0f} nT")
|
||||||
|
|
||||||
|
try:
|
||||||
|
out = write_calibrated(cap, args.output, args.csv, cal, factor,
|
||||||
|
args.calibration)
|
||||||
|
except (OSError, CalibrationError) as exc:
|
||||||
|
sys.exit(str(exc))
|
||||||
|
print(f"-> {out}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
|
|
@ -70,6 +70,13 @@ class Capture:
|
||||||
self.dt_true = float(slope)
|
self.dt_true = float(slope)
|
||||||
# Error in the *rate*, so the sign matches the reported Hz: negative
|
# Error in the *rate*, so the sign matches the reported Hz: negative
|
||||||
# means the chip samples slower than the nominal table value.
|
# means the chip samples slower than the nominal table value.
|
||||||
|
#
|
||||||
|
# Against the *nominal* model, which no particular part obeys. Its own
|
||||||
|
# error is cycle-count dependent -- on one unit, +0.84% at cycle count
|
||||||
|
# 100 and -0.99% at 400, same supply -- so this is a diagnostic of the
|
||||||
|
# configuration, not a measurement of the oscillator, and multiplying a
|
||||||
|
# field by (1 + rate_error) is not a gain correction. calibrate.py does
|
||||||
|
# that against a measured per-unit reference.
|
||||||
self.rate_error = self.dt_nominal / self.dt_true - 1.0
|
self.rate_error = self.dt_nominal / self.dt_true - 1.0
|
||||||
self.residuals = (system_time - time_0) - (
|
self.residuals = (system_time - time_0) - (
|
||||||
intercept + slope * (sample_index - index_0))
|
intercept + slope * (sample_index - index_0))
|
||||||
|
|
|
||||||
457
characterize.py
457
characterize.py
|
|
@ -15,6 +15,24 @@ Produces a four-panel figure and a text summary:
|
||||||
Read latency host-side diagnostics only -- the measurement
|
Read latency host-side diagnostics only -- the measurement
|
||||||
grid itself is uniform regardless.
|
grid itself is uniform regardless.
|
||||||
|
|
||||||
|
Three further figures are written beside it, each answering something the four
|
||||||
|
panels collapse away:
|
||||||
|
|
||||||
|
_spectrogram.png whether a feature lasted the run or was a burst.
|
||||||
|
_drift.png how far the chip's oscillator moved against the
|
||||||
|
host clock, and -- from the two-sample deviation
|
||||||
|
of the rate -- how much of that is real. These
|
||||||
|
parts drift 186-1241 ppm against a floor near
|
||||||
|
10 ppm, so it is a signal by a wide margin.
|
||||||
|
_timebase.png which clock each line is coherent on. A line at a
|
||||||
|
fixed frequency sharpens when the samples are
|
||||||
|
placed by the host clock; one locked to the
|
||||||
|
sampling is wrecked by it, which settles from a
|
||||||
|
single capture what compare.py needs two rates to
|
||||||
|
decide. Diagnostic only: every other figure here
|
||||||
|
uses the chip's grid, because neither time base is
|
||||||
|
right for both kinds of line.
|
||||||
|
|
||||||
Two noise figures are printed, and the difference between them matters. `sd` is
|
Two noise figures are printed, and the difference between them matters. `sd` is
|
||||||
the plain standard deviation, which on a drifting capture measures the drift:
|
the plain standard deviation, which on a drifting capture measures the drift:
|
||||||
one run here reads 167.9 nT of sd on an axis whose actual noise is 12.1 nT.
|
one run here reads 167.9 nT of sd on an axis whose actual noise is 12.1 nT.
|
||||||
|
|
@ -74,21 +92,32 @@ SPEC_ASD_NT = 1.2
|
||||||
# compared by eye. Autoscaling defeats that completely: a quieter capture simply
|
# compared by eye. Autoscaling defeats that completely: a quieter capture simply
|
||||||
# redraws its own axis and looks identical to a noisy one. The ranges are wide
|
# redraws its own axis and looks identical to a noisy one. The ranges are wide
|
||||||
# enough for any configuration this driver offers -- cycle counts 50 to 800, so
|
# enough for any configuration this driver offers -- cycle counts 50 to 800, so
|
||||||
# rates from ~2 to ~530 Hz and LSBs from 3 to 50 nT -- and are deliberately whole
|
# rates from ~2 to ~530 Hz and LSBs from 3 to 50 nT. Where an axis is
|
||||||
# decades so the tick labels are powers of ten.
|
# logarithmic the range is whole decades, so the tick labels are powers of ten.
|
||||||
ASD_HZ = (1e-2, 1e3) # 0.05 Hz is the lowest bin a 20 s segment gives
|
#
|
||||||
ASD_NT = (1e-1, 1e2) # floor ~1 nT/rtHz, lines to ~15
|
# The ASD frequency axis is the exception: linear, not logarithmic. A log axis
|
||||||
|
# gives the sub-hertz drift most of the width and crams every line worth naming
|
||||||
|
# -- mains, its harmonics, fs/4 -- into the last fifth. Linear spaces them
|
||||||
|
# evenly, which is how a spectrum is read for lines. It costs the low-frequency
|
||||||
|
# decades, which the Allan panel beside it covers better anyway, and it cuts off
|
||||||
|
# above 100 Hz: at cycle count 100 that leaves out 100-150 Hz, where nothing but
|
||||||
|
# the noise floor has ever appeared.
|
||||||
|
ASD_HZ = (0.0, 100.0)
|
||||||
|
ASD_NT = (5e-1, 5e1) # floor ~1 nT/rtHz, lines to ~15
|
||||||
ALLAN_TAU_S = (1e-3, 1e3) # one sample period to a 1000 s average
|
ALLAN_TAU_S = (1e-3, 1e3) # one sample period to a 1000 s average
|
||||||
ALLAN_NT = (1e-1, 1e2)
|
ALLAN_NT = (5e-1, 5e1)
|
||||||
RESIDUAL_NT = (-150.0, 150.0) # +/-6 LSB at cycle count 100
|
RESIDUAL_NT = (-150.0, 150.0) # +/-6 LSB at cycle count 100
|
||||||
RESIDUAL_DENSITY = (0.0, 0.045)
|
RESIDUAL_DENSITY = (0.0, 0.045)
|
||||||
LATENCY_MS = (0.0, 20.0) # covers 1.9 ms at cc=50 to 13.7 ms at cc=400
|
LATENCY_MS = (0.0, 20.0) # covers 1.9 ms at cc=50 to 13.7 ms at cc=400
|
||||||
LATENCY_COUNT = (0.8, 1e5)
|
LATENCY_COUNT = (0.8, 1e5)
|
||||||
|
|
||||||
# compare.py plots the same two quantities normalised by |B|, so they get their
|
# compare.py plots the same two quantities normalised by |B|, so they get their
|
||||||
# own fixed ranges in ppm. Same reasoning: two invocations should overlay.
|
# own fixed ranges in ppm. Same reasoning: two invocations should overlay. Its
|
||||||
|
# frequency axis stays logarithmic: that figure overlays captures whose rates
|
||||||
|
# differ several-fold, and log is what puts them on comparable footing.
|
||||||
FRACTIONAL_ASD_PPM = (1e0, 1e3)
|
FRACTIONAL_ASD_PPM = (1e0, 1e3)
|
||||||
FRACTIONAL_ALLAN_PPM = (1e0, 1e3)
|
FRACTIONAL_ALLAN_PPM = (1e0, 1e3)
|
||||||
|
FRACTIONAL_ASD_HZ = (1e-2, 1e3)
|
||||||
|
|
||||||
# Spectrogram segment length. Longer than SEGMENT_SECONDS because the job here
|
# Spectrogram segment length. Longer than SEGMENT_SECONDS because the job here
|
||||||
# is to resolve one line from another -- 40 s gives 0.025 Hz, enough to separate
|
# is to resolve one line from another -- 40 s gives 0.025 Hz, enough to separate
|
||||||
|
|
@ -169,6 +198,50 @@ SEGMENT_SECONDS = 20.0
|
||||||
# the ~20 fractions tested. Real artefacts here reach 30.
|
# the ~20 fractions tested. Real artefacts here reach 30.
|
||||||
LINE_SIGMA = 5.0
|
LINE_SIGMA = 5.0
|
||||||
|
|
||||||
|
# Drift figure. The left panel fits the rate on windows of DRIFT_WINDOW_S; the
|
||||||
|
# right sweeps that window length to show what it costs. 10 s is a compromise
|
||||||
|
# and not the optimum -- sigma_y bottoms out nearer 2 s on these parts -- but 2 s
|
||||||
|
# windows put 145 points on a 290 s capture, which draws as a band rather than a
|
||||||
|
# trend. At 10 s there are ~29, the floor is still only 13-34 ppm, and the drift
|
||||||
|
# being plotted is 186-1241 ppm, so the error bars stay a minor part of the span.
|
||||||
|
DRIFT_WINDOW_S = 10.0
|
||||||
|
# Log-spaced, from a window that holds a few hundred samples to a third of a
|
||||||
|
# 300 s capture -- past that there are fewer than three windows and `rate_allan`
|
||||||
|
# drops the point rather than estimate a deviation from one difference.
|
||||||
|
RATE_TAUS = (1.0, 1.5, 2.0, 3.0, 5.0, 7.0, 10.0, 15.0,
|
||||||
|
20.0, 30.0, 50.0, 70.0, 100.0)
|
||||||
|
DRIFT_PPM = (-800.0, 800.0) # the worst run here spans 1241 ppm peak to peak
|
||||||
|
RATE_TAU_S = (1.0, 1e2)
|
||||||
|
RATE_ALLAN_PPM = (1e0, 1e3)
|
||||||
|
|
||||||
|
# Time-base figure. The host clock is fitted against sample index by a
|
||||||
|
# polynomial of this degree: high enough to follow a warm-up transient, low
|
||||||
|
# enough that it cannot chase the per-read scheduling jitter, which carries no
|
||||||
|
# information about the chip. Degree 1 is exactly the uniform grid already in
|
||||||
|
# use, and nothing above 3 changed any measured amplitude on these captures --
|
||||||
|
# consistent with `calibrate.rate_allan`, which finds nothing above the noise
|
||||||
|
# below a ~2 s timescale.
|
||||||
|
TIMEBASE_DEGREE = 3
|
||||||
|
# Half-width of the scan around each line, and how finely it is sampled. A
|
||||||
|
# 290 s capture resolves 1/T = 0.0034 Hz, so +/-0.05 Hz is ~30 resolution widths
|
||||||
|
# -- wide enough to show the shape of a peak and whether it moved, narrow enough
|
||||||
|
# that the peak is not a single pixel.
|
||||||
|
TIMEBASE_SPAN_HZ = 0.05
|
||||||
|
TIMEBASE_POINTS = 401
|
||||||
|
TIMEBASE_PANELS = 3
|
||||||
|
# A verdict needs a peak that is actually a peak: this is how far the scan's
|
||||||
|
# maximum must stand above the scan's own median before the ratio of the two
|
||||||
|
# time bases means anything.
|
||||||
|
#
|
||||||
|
# Local contrast rather than significance against the broadband noise, because
|
||||||
|
# that is the question. A line can sit at a respectable sigma across the whole
|
||||||
|
# band and still have no resolvable peak in a 0.1 Hz window -- the 60 Hz alias
|
||||||
|
# in |B| at cycle count 400 does exactly that, ranking third at 9 sigma while
|
||||||
|
# its scan is visibly flat -- and comparing where two flat curves happen to peak
|
||||||
|
# is comparing noise. 3x is a little above what a smooth curve reaches by
|
||||||
|
# chance and well below the 4-6x a real line here shows.
|
||||||
|
TIMEBASE_MIN_CONTRAST = 3.0
|
||||||
|
|
||||||
|
|
||||||
def band_for(fs):
|
def band_for(fs):
|
||||||
"""The (lo, hi) analysis band for a capture sampled at fs."""
|
"""The (lo, hi) analysis band for a capture sampled at fs."""
|
||||||
|
|
@ -528,6 +601,71 @@ def reference_lines(fs):
|
||||||
return sorted(marks)
|
return sorted(marks)
|
||||||
|
|
||||||
|
|
||||||
|
def coherent_amplitude(v, t, freqs):
|
||||||
|
"""Amplitude of a tone at each of `freqs`, for samples taken at times `t`.
|
||||||
|
|
||||||
|
Evaluates 2*|mean(v * exp(-2*pi*i*f*t))| directly. That is what a spectrum
|
||||||
|
computes, but without requiring the samples to be evenly spaced, which is
|
||||||
|
the entire reason this exists rather than a call to `welch_asd`: the
|
||||||
|
question here is which *time base* a line is coherent on, and one of the two
|
||||||
|
candidates is not uniform.
|
||||||
|
|
||||||
|
A tone present at f throughout the record sums in phase and returns its own
|
||||||
|
amplitude; anything else averages towards zero as 1/sqrt(n). A sample placed
|
||||||
|
at the wrong time by a fraction d of a cycle contributes cos(2*pi*d) instead
|
||||||
|
of 1, so coherence is lost at a rate set by the tone's frequency -- which
|
||||||
|
makes this the sharpest test available of whether a time base is right, and
|
||||||
|
also the reason a low-frequency line can say nothing about one.
|
||||||
|
|
||||||
|
The mean is removed first, or a DC offset leaks into the lowest frequencies.
|
||||||
|
"""
|
||||||
|
v = np.asarray(v, dtype=float)
|
||||||
|
v = v - v.mean()
|
||||||
|
t = np.asarray(t, dtype=float)
|
||||||
|
return np.array([2.0 * abs(complex((v * np.exp(-2j * np.pi * f * t)).mean()))
|
||||||
|
for f in np.atleast_1d(freqs)])
|
||||||
|
|
||||||
|
|
||||||
|
def host_time_base(cap, degree=TIMEBASE_DEGREE):
|
||||||
|
"""Sample times taken from the host clock, smoothed against sample index.
|
||||||
|
|
||||||
|
The alternative to `cap.elapsed`, which is `sample_index * dt_true` and so
|
||||||
|
asserts the rate never moved. It did: `calibrate.rate_stability` measures
|
||||||
|
186 to 1241 ppm of movement across the captures here.
|
||||||
|
|
||||||
|
Raw host timestamps carry that drift, but also the scheduling jitter of each
|
||||||
|
individual read, which says nothing about when the chip actually sampled.
|
||||||
|
A low-order polynomial keeps the first and discards the second. Degree 1
|
||||||
|
reduces exactly to `cap.elapsed`, which makes the uniform grid the degree-1
|
||||||
|
member of this family rather than a separate idea, and nothing above degree
|
||||||
|
3 changed any measured amplitude on these captures.
|
||||||
|
|
||||||
|
Returned relative to the first sample, like `cap.elapsed - cap.elapsed[0]`.
|
||||||
|
Shifting to that origin is what makes the degree-1 case *exactly* the
|
||||||
|
uniform grid rather than merely parallel to it: the fit carries a non-zero
|
||||||
|
intercept, since a least-squares line through a curve does not pass through
|
||||||
|
its first point.
|
||||||
|
"""
|
||||||
|
index = cap.sample_index - cap.sample_index[0]
|
||||||
|
host = cap.system_time - cap.system_time[0]
|
||||||
|
if len(index) <= degree + 1:
|
||||||
|
return index * cap.dt_true
|
||||||
|
fitted = np.polyval(np.polyfit(index, host, degree), index)
|
||||||
|
return fitted - fitted[0]
|
||||||
|
|
||||||
|
|
||||||
|
def sibling_path(path, suffix):
|
||||||
|
"""Name a companion figure from the main one: `x_noise.png` -> `x_drift.png`.
|
||||||
|
|
||||||
|
The trailing `_noise` is stripped first, so the extra figures do not come
|
||||||
|
out named `_noise_drift` and `_noise_spectrogram`.
|
||||||
|
"""
|
||||||
|
stem = path.rsplit(".", 1)[0]
|
||||||
|
if stem.endswith("_noise"):
|
||||||
|
stem = stem[:-len("_noise")]
|
||||||
|
return f"{stem}_{suffix}.png"
|
||||||
|
|
||||||
|
|
||||||
def make_spectrogram(cap, path, seconds=SPECTROGRAM_SECONDS,
|
def make_spectrogram(cap, path, seconds=SPECTROGRAM_SECONDS,
|
||||||
overlap=SPECTROGRAM_OVERLAP,
|
overlap=SPECTROGRAM_OVERLAP,
|
||||||
colormap=SPECTROGRAM_COLORMAP, max_freq=None):
|
colormap=SPECTROGRAM_COLORMAP, max_freq=None):
|
||||||
|
|
@ -535,6 +673,8 @@ def make_spectrogram(cap, path, seconds=SPECTROGRAM_SECONDS,
|
||||||
fs = cap.true_rate_hz
|
fs = cap.true_rate_hz
|
||||||
times, freqs, asd = spectrogram(cap.total, fs, seconds, overlap)
|
times, freqs, asd = spectrogram(cap.total, fs, seconds, overlap)
|
||||||
native = freqs[1] - freqs[0]
|
native = freqs[1] - freqs[0]
|
||||||
|
import calibrate
|
||||||
|
rate_drift = calibrate.rate_stability(cap)
|
||||||
freqs, asd = bin_frequency(freqs, asd)
|
freqs, asd = bin_frequency(freqs, asd)
|
||||||
# Fractional, so a gain difference between captures is not read as a noise
|
# Fractional, so a gain difference between captures is not read as a noise
|
||||||
# difference when they share a colour scale.
|
# difference when they share a colour scale.
|
||||||
|
|
@ -610,6 +750,8 @@ def make_spectrogram(cap, path, seconds=SPECTROGRAM_SECONDS,
|
||||||
f"at {fs:.2f} Hz, cycle count {cap.cycle_count}",
|
f"at {fs:.2f} Hz, cycle count {cap.cycle_count}",
|
||||||
color=TEXT_PRIMARY, fontsize=10, ha="center")
|
color=TEXT_PRIMARY, fontsize=10, ha="center")
|
||||||
fig.text(0.5, 0.935,
|
fig.text(0.5, 0.935,
|
||||||
|
f"oscillator drifted {rate_drift * 1e6:.0f} ppm across the run — "
|
||||||
|
f"a line at fs/4 smears by {rate_drift * fs / 4:.4f} Hz. "
|
||||||
f"{seconds:g} s windows, {native:.3f} Hz native binned to "
|
f"{seconds:g} s windows, {native:.3f} Hz native binned to "
|
||||||
f"{SPECTROGRAM_BIN_HZ:g} Hz ({dof} dof/cell), {overlap:.2%} overlap, "
|
f"{SPECTROGRAM_BIN_HZ:g} Hz ({dof} dof/cell), {overlap:.2%} overlap, "
|
||||||
f"{len(times)} columns from {independent} independent windows. "
|
f"{len(times)} columns from {independent} independent windows. "
|
||||||
|
|
@ -622,6 +764,239 @@ def make_spectrogram(cap, path, seconds=SPECTROGRAM_SECONDS,
|
||||||
return path
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def make_drift(cap, path, window=DRIFT_WINDOW_S):
|
||||||
|
"""Write the oscillator's drift against the host clock, with its error bar.
|
||||||
|
|
||||||
|
Two views of one measurement. The left panel is the chip's rate through the
|
||||||
|
run; the right is how well that rate can be known as a function of how long
|
||||||
|
it is measured for -- which is what decides whether the left panel shows a
|
||||||
|
signal or its own noise. On these parts it is emphatically a signal: 186 to
|
||||||
|
1241 ppm of drift against a floor near 10 ppm.
|
||||||
|
"""
|
||||||
|
import calibrate
|
||||||
|
centres, rates = calibrate.window_rates(cap, window)
|
||||||
|
if len(rates) < 3:
|
||||||
|
raise ValueError(f"{cap.duration:.0f} s gives {len(rates)} window(s) of "
|
||||||
|
f"{window:g} s, too few to show a trend")
|
||||||
|
taus, devs = calibrate.rate_allan(cap, RATE_TAUS)
|
||||||
|
if not len(taus):
|
||||||
|
raise ValueError(f"{cap.duration:.0f} s is too short for any averaging "
|
||||||
|
f"time in {RATE_TAUS[0]:g}-{RATE_TAUS[-1]:g} s")
|
||||||
|
# The error bar is the two-sample deviation at the window length actually
|
||||||
|
# plotted. Not the scatter of the points themselves, which is the drift this
|
||||||
|
# figure exists to show, and using it would beg the question.
|
||||||
|
floor = float(np.interp(window, taus, devs))
|
||||||
|
best = int(np.argmin(devs))
|
||||||
|
ppm = (rates / rates.mean() - 1.0) * 1e6
|
||||||
|
span = calibrate.rate_stability(cap) * 1e6
|
||||||
|
|
||||||
|
fig, axs = plt.subplots(1, 2, figsize=(13.5, 5.5), dpi=150)
|
||||||
|
fig.patch.set_facecolor(SURFACE)
|
||||||
|
for ax in axs:
|
||||||
|
ax.set_facecolor(SURFACE)
|
||||||
|
|
||||||
|
a = axs[0]
|
||||||
|
a.axhline(0.0, color=REFERENCE, linestyle="--", linewidth=1.0)
|
||||||
|
a.errorbar(centres, ppm, yerr=floor * 1e6, fmt="o", markersize=3.5,
|
||||||
|
color=AXES[0][2], ecolor=REFERENCE, elinewidth=1.0, capsize=2.5,
|
||||||
|
label=f"{window:g} s windows, ± σ_y = {floor * 1e6:.0f} ppm")
|
||||||
|
trend = np.polyfit(centres, ppm, 1)
|
||||||
|
a.plot(centres, np.polyval(trend, centres), color=TEXT_PRIMARY, linewidth=1.5,
|
||||||
|
label=f"trend {trend[0] * cap.duration:+,.0f} ppm across the run")
|
||||||
|
a.set_title("Chip oscillator against the host clock", loc="left",
|
||||||
|
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
||||||
|
a.set_xlabel("elapsed (s)")
|
||||||
|
a.set_ylabel("rate error (ppm of the run mean)")
|
||||||
|
# The y axis is fixed so two runs can be compared; the x axis is the
|
||||||
|
# capture's own length, which pinning would simply clip on a longer one.
|
||||||
|
a.set_ylim(*DRIFT_PPM)
|
||||||
|
a.annotate("the sample grid stays uniform in index —\nthis is its spacing "
|
||||||
|
"in seconds moving.\nGain does not follow: it is set by the "
|
||||||
|
"ratio\nof two on-die clocks, which drift together.",
|
||||||
|
xy=(0.02, 0.05), xycoords="axes fraction",
|
||||||
|
color=TEXT_SECONDARY, fontsize=9)
|
||||||
|
|
||||||
|
a = axs[1]
|
||||||
|
a.loglog(taus, devs * 1e6, "o-", color=AXES[0][2], linewidth=1.5,
|
||||||
|
markersize=4.5, label="σ_y(τ), measured")
|
||||||
|
a.plot([taus[best]], [devs[best] * 1e6], "o", markersize=11,
|
||||||
|
markerfacecolor="none", markeredgecolor=TEXT_PRIMARY,
|
||||||
|
markeredgewidth=1.4, zorder=5)
|
||||||
|
a.annotate(f"{devs[best] * 1e6:.1f} ppm at τ = {taus[best]:g} s",
|
||||||
|
xy=(taus[best], devs[best] * 1e6), xytext=(0, -26),
|
||||||
|
textcoords="offset points", ha="center",
|
||||||
|
color=TEXT_PRIMARY, fontsize=9)
|
||||||
|
# Anchored at the longest tau and drawn back to the minimum: white noise
|
||||||
|
# cannot rise, so a branch parallel to this is a frequency ramp, and one
|
||||||
|
# steeper than it is not.
|
||||||
|
rising = taus[best:]
|
||||||
|
a.plot(rising, devs[-1] * 1e6 * rising / taus[-1], linestyle=":",
|
||||||
|
color=REFERENCE, linewidth=1.5,
|
||||||
|
label="∝ τ — a deterministic frequency ramp")
|
||||||
|
a.set_title("How well the rate can be measured", loc="left",
|
||||||
|
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
||||||
|
a.set_xlabel("averaging time τ (s)"); a.set_ylabel("σ_y (ppm)")
|
||||||
|
a.set_xlim(*RATE_TAU_S); a.set_ylim(*RATE_ALLAN_PPM)
|
||||||
|
decade_ticks(a)
|
||||||
|
a.annotate("falling = measurement noise averaging down;\nrising = drift the "
|
||||||
|
"measurement has resolved",
|
||||||
|
xy=(0.02, 0.05), xycoords="axes fraction",
|
||||||
|
color=TEXT_SECONDARY, fontsize=9)
|
||||||
|
|
||||||
|
for ax in axs:
|
||||||
|
ax.grid(True, which="both", color=GRID, linewidth=0.7)
|
||||||
|
ax.set_axisbelow(True)
|
||||||
|
for side in ("top", "right"):
|
||||||
|
ax.spines[side].set_visible(False)
|
||||||
|
for side in ("left", "bottom"):
|
||||||
|
ax.spines[side].set_color(GRID)
|
||||||
|
ax.tick_params(colors=TEXT_SECONDARY, labelsize=9, length=0)
|
||||||
|
ax.xaxis.label.set_color(TEXT_SECONDARY)
|
||||||
|
ax.yaxis.label.set_color(TEXT_SECONDARY)
|
||||||
|
ax.legend(frameon=False, fontsize=9, labelcolor=TEXT_SECONDARY,
|
||||||
|
loc="upper right")
|
||||||
|
|
||||||
|
fig.text(0.5, 0.955,
|
||||||
|
f"{cap.path.rsplit('/', 1)[-1]} — {len(cap.sample_index):,} samples "
|
||||||
|
f"at {cap.true_rate_hz:.2f} Hz, cycle count {cap.cycle_count}",
|
||||||
|
color=TEXT_PRIMARY, fontsize=10, ha="center")
|
||||||
|
# Two figures for the drift, deliberately: the peak-to-peak spread is what
|
||||||
|
# `rate_stability` feeds to the uncertainty propagation and what the rest of
|
||||||
|
# the tooling quotes, while the fitted trend uses every window and is the
|
||||||
|
# better estimate of the warm-up itself. They differ by how much of the
|
||||||
|
# transient a coarse window averages over, and printing only one invites the
|
||||||
|
# other to be read off the legend and mistaken for a discrepancy.
|
||||||
|
fig.text(0.5, 0.915,
|
||||||
|
f"drifted {span:.0f} ppm peak to peak, "
|
||||||
|
f"{abs(trend[0] * cap.duration):.0f} ppm by the fitted trend, "
|
||||||
|
f"against a floor of {devs[best] * 1e6:.1f} ppm at τ = "
|
||||||
|
f"{taus[best]:g} s — resolved {span / (devs[best] * 1e6):.0f}× over.",
|
||||||
|
color=TEXT_SECONDARY, fontsize=9, ha="center")
|
||||||
|
fig.tight_layout(rect=[0, 0, 1, 0.90])
|
||||||
|
fig.savefig(path, facecolor=SURFACE)
|
||||||
|
plt.close(fig)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def make_time_base(cap, path):
|
||||||
|
"""Write, line by line, which clock each one is coherent on.
|
||||||
|
|
||||||
|
A line at a fixed frequency lives in wall time, so it sharpens when the
|
||||||
|
samples are placed by the host clock. A line locked to the sampling lives in
|
||||||
|
sample index, and the same substitution wrecks it. Measuring both ways
|
||||||
|
therefore separates a real signal from an artefact of measuring *within a
|
||||||
|
single capture*, where compare.py needs two at different rates to do it.
|
||||||
|
|
||||||
|
The catch is that the test only has force on a strong line: coherence is
|
||||||
|
lost in proportion to the tone's frequency, so a low-frequency or aliased
|
||||||
|
line says little, and a weak one returns noise. Both are marked as such
|
||||||
|
rather than being given a verdict they cannot support.
|
||||||
|
"""
|
||||||
|
fs = cap.true_rate_hz
|
||||||
|
chip_t = (cap.sample_index - cap.sample_index[0]) * cap.dt_true
|
||||||
|
host_t = host_time_base(cap)
|
||||||
|
# The same cubic detrend `sample_locked_lines` applies, and for the same
|
||||||
|
# reason: drift is not a line, and left in place it leaks into everything
|
||||||
|
# below a few tenths of a hertz.
|
||||||
|
order = np.arange(len(cap.total))
|
||||||
|
v = cap.total - np.polyval(np.polyfit(order, cap.total, 3), order)
|
||||||
|
# Against the quantiser rather than against zero. A capture whose |B| never
|
||||||
|
# moves by even a thousandth of an LSB has no spectrum -- every sample is
|
||||||
|
# the same integer -- but the cubic detrend still leaves ~1e-11 nT of
|
||||||
|
# floating-point residue, so an exact test would let it through and then
|
||||||
|
# find "lines" in rounding error.
|
||||||
|
if v.std() < cap.lsb_nt * 1e-3:
|
||||||
|
raise ValueError("|B| is constant to within a thousandth of an LSB; "
|
||||||
|
"no lines to test")
|
||||||
|
null = v.std() * np.sqrt(2 / len(v))
|
||||||
|
|
||||||
|
candidates = [(line.numerator / line.period * fs,
|
||||||
|
f"{line.numerator}/{line.period} of fs", False)
|
||||||
|
for line in sample_locked_lines(v)]
|
||||||
|
candidates += [(hz, label, folded)
|
||||||
|
for hz, label, folded in reference_lines(fs)
|
||||||
|
if not label.startswith("fs/")]
|
||||||
|
|
||||||
|
scored = []
|
||||||
|
for hz, label, folded in candidates:
|
||||||
|
if not 0 < hz < fs / 2:
|
||||||
|
continue
|
||||||
|
sigma = float(coherent_amplitude(v, chip_t, hz)[0]) / null
|
||||||
|
scored.append((sigma, hz, label, folded))
|
||||||
|
scored.sort(reverse=True)
|
||||||
|
scored = scored[:TIMEBASE_PANELS]
|
||||||
|
if not scored:
|
||||||
|
raise ValueError("no lines to compare time bases on")
|
||||||
|
|
||||||
|
# Floored rather than purely per-panel: a capture with one or two lines
|
||||||
|
# worth testing would otherwise be narrower than its own caption.
|
||||||
|
fig, axs = plt.subplots(1, len(scored),
|
||||||
|
figsize=(max(9.8, 4.7 * len(scored)), 5.2),
|
||||||
|
dpi=150, squeeze=False)
|
||||||
|
fig.patch.set_facecolor(SURFACE)
|
||||||
|
for ax, (sigma, hz, label, folded) in zip(axs[0], scored):
|
||||||
|
ax.set_facecolor(SURFACE)
|
||||||
|
freqs = np.linspace(hz - TIMEBASE_SPAN_HZ, hz + TIMEBASE_SPAN_HZ,
|
||||||
|
TIMEBASE_POINTS)
|
||||||
|
chip = coherent_amplitude(v, chip_t, freqs)
|
||||||
|
host = coherent_amplitude(v, host_t, freqs)
|
||||||
|
ratio = host.max() / chip.max()
|
||||||
|
contrast = chip.max() / np.median(chip)
|
||||||
|
if folded:
|
||||||
|
verdict = "aliased — folding assumes uniformity, not a clean test"
|
||||||
|
elif contrast < TIMEBASE_MIN_CONTRAST:
|
||||||
|
verdict = f"no resolved peak ({contrast:.1f}× its floor) — no verdict"
|
||||||
|
elif ratio > 1.05:
|
||||||
|
verdict = "coherent in wall time — external"
|
||||||
|
elif ratio < 0.95:
|
||||||
|
verdict = "coherent in sample index — an artefact of measuring"
|
||||||
|
else:
|
||||||
|
verdict = "no preference between the two"
|
||||||
|
|
||||||
|
ax.plot(freqs - hz, chip, color=AXES[0][2], linewidth=1.3,
|
||||||
|
label=f"chip index grid — {chip.max():.3f} nT")
|
||||||
|
ax.plot(freqs - hz, host, color=AXES[1][2], linewidth=1.3,
|
||||||
|
label=f"host clock, degree {TIMEBASE_DEGREE} — {host.max():.3f} nT")
|
||||||
|
# Padded so the verdict can sit between title and axes without either
|
||||||
|
# landing on the traces, which reach the top of a tightly scaled panel.
|
||||||
|
ax.set_title(f"{label} at {hz:.3f} Hz {ratio:.2f}×", loc="left",
|
||||||
|
color=TEXT_PRIMARY, fontsize=11, fontweight="bold", pad=22)
|
||||||
|
ax.annotate(verdict, xy=(0.0, 1.01), xycoords="axes fraction",
|
||||||
|
color=TEXT_SECONDARY, fontsize=8.5, va="bottom")
|
||||||
|
ax.set_xlabel(f"offset from {hz:.3f} Hz (Hz)")
|
||||||
|
ax.set_ylabel("coherent amplitude (nT)")
|
||||||
|
ax.set_xlim(-TIMEBASE_SPAN_HZ, TIMEBASE_SPAN_HZ)
|
||||||
|
# Headroom for the legend, which would otherwise sit on the peak the
|
||||||
|
# panel exists to show.
|
||||||
|
ax.set_ylim(0.0, max(chip.max(), host.max()) * 1.45)
|
||||||
|
ax.grid(True, which="both", color=GRID, linewidth=0.7)
|
||||||
|
ax.set_axisbelow(True)
|
||||||
|
for side in ("top", "right"):
|
||||||
|
ax.spines[side].set_visible(False)
|
||||||
|
for side in ("left", "bottom"):
|
||||||
|
ax.spines[side].set_color(GRID)
|
||||||
|
ax.tick_params(colors=TEXT_SECONDARY, labelsize=9, length=0)
|
||||||
|
ax.xaxis.label.set_color(TEXT_SECONDARY)
|
||||||
|
ax.yaxis.label.set_color(TEXT_SECONDARY)
|
||||||
|
ax.legend(frameon=False, fontsize=8.5, labelcolor=TEXT_SECONDARY,
|
||||||
|
loc="upper right")
|
||||||
|
|
||||||
|
fig.suptitle("Which clock is each line coherent on?", color=TEXT_PRIMARY,
|
||||||
|
fontsize=13, fontweight="bold", y=0.99)
|
||||||
|
fig.text(0.5, 0.930,
|
||||||
|
f"{cap.path.rsplit('/', 1)[-1]} at {fs:.2f} Hz, "
|
||||||
|
f"cycle count {cap.cycle_count}",
|
||||||
|
color=TEXT_SECONDARY, fontsize=9, ha="center")
|
||||||
|
fig.text(0.5, 0.897,
|
||||||
|
"Ratio is host peak over chip peak. Diagnostic only — no other "
|
||||||
|
"figure uses the host time base.",
|
||||||
|
color=TEXT_SECONDARY, fontsize=9, ha="center")
|
||||||
|
fig.tight_layout(rect=[0, 0, 1, 0.88])
|
||||||
|
fig.savefig(path, facecolor=SURFACE)
|
||||||
|
plt.close(fig)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
def trimmed(cap, seconds):
|
def trimmed(cap, seconds):
|
||||||
"""Drop `seconds` from both ends of a capture. Returns (capture, note).
|
"""Drop `seconds` from both ends of a capture. Returns (capture, note).
|
||||||
|
|
||||||
|
|
@ -720,6 +1095,19 @@ def main():
|
||||||
print(cap.summary())
|
print(cap.summary())
|
||||||
if trim_note:
|
if trim_note:
|
||||||
print(f" {trim_note}")
|
print(f" {trim_note}")
|
||||||
|
# How far the oscillator moved during the run. Imported here rather than at
|
||||||
|
# module scope: calibrate.py imports this module, so the dependency only
|
||||||
|
# runs one way at import time.
|
||||||
|
import calibrate
|
||||||
|
drift = calibrate.rate_stability(cap)
|
||||||
|
# Reported, not flagged. The sensor oscillator is an LR relaxation circuit
|
||||||
|
# -- coil inductance, a resistor and comparator thresholds, none of them
|
||||||
|
# temperature-stable -- so a tenth of a percent across a run is ordinary,
|
||||||
|
# not a fault. What it is, is the error bar on this capture's rate and on
|
||||||
|
# anything derived from it.
|
||||||
|
print(f" oscillator drifted {drift * 1e6:.0f} ppm across the run "
|
||||||
|
f"({drift * cap.dt_true * 1e6:.2f} us on a "
|
||||||
|
f"{cap.dt_true * 1e3:.2f} ms period) -- the error bar on the rate")
|
||||||
print()
|
print()
|
||||||
print(f"{'series':6s} {'sd':>9s} {'white sd':>9s} {'dither':>7s} "
|
print(f"{'series':6s} {'sd':>9s} {'white sd':>9s} {'dither':>7s} "
|
||||||
f"{'sd/|B|':>10s} {'p2p':>9s} {'vs spec':>8s} {'ASD':>7s}")
|
f"{'sd/|B|':>10s} {'p2p':>9s} {'vs spec':>8s} {'ASD':>7s}")
|
||||||
|
|
@ -737,7 +1125,10 @@ def main():
|
||||||
wsd = white_sd(v)
|
wsd = white_sd(v)
|
||||||
|
|
||||||
freqs, asd = welch_asd(v, fs)
|
freqs, asd = welch_asd(v, fs)
|
||||||
axs[0, 0].loglog(freqs, asd, color=color, linewidth=1.2,
|
# Thin: four traces overlaid on a linear 0-100 Hz axis, and the detail
|
||||||
|
# between the lines is what the panel is read for. The Allan panel below
|
||||||
|
# keeps a heavier stroke -- four smooth curves, not four dense ones.
|
||||||
|
axs[0, 0].semilogy(freqs, asd, color=color, linewidth=0.7,
|
||||||
label=label, alpha=0.85)
|
label=label, alpha=0.85)
|
||||||
|
|
||||||
taus, devs = allan_deviation(v, fs)
|
taus, devs = allan_deviation(v, fs)
|
||||||
|
|
@ -804,13 +1195,11 @@ def main():
|
||||||
# The band the quoted median comes from, so the number and the picture
|
# The band the quoted median comes from, so the number and the picture
|
||||||
# cannot drift apart. Drawn under the traces, hence the low zorder.
|
# cannot drift apart. Drawn under the traces, hence the low zorder.
|
||||||
a.axvspan(band[0], band[1], color=GRID, alpha=0.55, zorder=0, linewidth=0)
|
a.axvspan(band[0], band[1], color=GRID, alpha=0.55, zorder=0, linewidth=0)
|
||||||
a.axhline(SPEC_ASD_NT, color=REFERENCE, linestyle="--", linewidth=1.2)
|
# Named in the legend rather than annotated on the line. The spec sits low
|
||||||
# Anchored to the left margin of the fixed scale, which no capture reaches:
|
# in the range, where on a linear axis the traces are densest, and floating
|
||||||
# the lowest bin a 20 s segment can produce is 0.05 Hz.
|
# text disappears into them wherever it is put.
|
||||||
a.annotate(f"Table 3-1 spec {SPEC_ASD_NT} nT/√Hz",
|
a.axhline(SPEC_ASD_NT, color=REFERENCE, linestyle="--", linewidth=1.2,
|
||||||
xy=(ASD_HZ[0] * 1.2, SPEC_ASD_NT),
|
label=f"Table 3-1 spec, {SPEC_ASD_NT} nT/√Hz")
|
||||||
xytext=(0, 5), textcoords="offset points",
|
|
||||||
color=REFERENCE, fontsize=9)
|
|
||||||
a.set_title("Amplitude spectral density", loc="left",
|
a.set_title("Amplitude spectral density", loc="left",
|
||||||
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
||||||
a.annotate(f"shaded: {band[0]:g}–{band[1]:.0f} Hz, the quoted band",
|
a.annotate(f"shaded: {band[0]:g}–{band[1]:.0f} Hz, the quoted band",
|
||||||
|
|
@ -818,7 +1207,7 @@ def main():
|
||||||
color=TEXT_SECONDARY, fontsize=9)
|
color=TEXT_SECONDARY, fontsize=9)
|
||||||
a.set_xlabel("frequency (Hz)"); a.set_ylabel("nT/√Hz")
|
a.set_xlabel("frequency (Hz)"); a.set_ylabel("nT/√Hz")
|
||||||
a.set_xlim(*ASD_HZ); a.set_ylim(*ASD_NT)
|
a.set_xlim(*ASD_HZ); a.set_ylim(*ASD_NT)
|
||||||
decade_ticks(a)
|
decade_ticks(a, which="y")
|
||||||
|
|
||||||
a = axs[0, 1]
|
a = axs[0, 1]
|
||||||
a.set_title("Allan deviation", loc="left",
|
a.set_title("Allan deviation", loc="left",
|
||||||
|
|
@ -833,13 +1222,6 @@ def main():
|
||||||
a = axs[1, 0]
|
a = axs[1, 0]
|
||||||
a.set_title("Residual distribution (cubic trend removed)", loc="left",
|
a.set_title("Residual distribution (cubic trend removed)", loc="left",
|
||||||
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
||||||
a.annotate(f"bins are {lsb_nt / 4:.2f} nT, a quarter LSB.\n"
|
|
||||||
f"Integer counts make this a comb at {lsb_nt:.1f} nT however\n"
|
|
||||||
f"well dithered; how sharp it looks only tracks how far\n"
|
|
||||||
f"the axis drifted. Whether averaging still beats the\n"
|
|
||||||
f"quantiser is the dither figure printed above.",
|
|
||||||
xy=(0.02, 0.97), xycoords="axes fraction", va="top",
|
|
||||||
color=TEXT_SECONDARY, fontsize=9)
|
|
||||||
a.set_xlabel("nT"); a.set_ylabel("density")
|
a.set_xlabel("nT"); a.set_ylabel("density")
|
||||||
a.set_xlim(*RESIDUAL_NT); a.set_ylim(*RESIDUAL_DENSITY)
|
a.set_xlim(*RESIDUAL_NT); a.set_ylim(*RESIDUAL_DENSITY)
|
||||||
|
|
||||||
|
|
@ -854,7 +1236,9 @@ def main():
|
||||||
color=TEXT_PRIMARY, fontsize=9)
|
color=TEXT_PRIMARY, fontsize=9)
|
||||||
a.set_title("Host read latency", loc="left",
|
a.set_title("Host read latency", loc="left",
|
||||||
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
color=TEXT_PRIMARY, fontsize=12, fontweight="bold", pad=8)
|
||||||
a.annotate("spread is host-side only;\nthe sample grid is exact",
|
# The oscillator's own drift used to be annotated here, on a panel about
|
||||||
|
# host-side comms that has nothing to do with it. It has its own figure now.
|
||||||
|
a.annotate("spread is comms only;\nthe sample grid is exact",
|
||||||
xy=(0.98, 0.94), xycoords="axes fraction", ha="right", va="top",
|
xy=(0.98, 0.94), xycoords="axes fraction", ha="right", va="top",
|
||||||
color=TEXT_SECONDARY, fontsize=9)
|
color=TEXT_SECONDARY, fontsize=9)
|
||||||
a.set_xlabel("interval between reads (ms)"); a.set_ylabel("count")
|
a.set_xlabel("interval between reads (ms)"); a.set_ylabel("count")
|
||||||
|
|
@ -883,7 +1267,7 @@ def main():
|
||||||
f"{len(t):,} samples, {cap.duration:.1f} s at {fs:.2f} Hz measured "
|
f"{len(t):,} samples, {cap.duration:.1f} s at {fs:.2f} Hz measured "
|
||||||
f"(nominal {cap.nominal_rate_hz:g} Hz, {cap.rate_error * 100:+.1f}%), "
|
f"(nominal {cap.nominal_rate_hz:g} Hz, {cap.rate_error * 100:+.1f}%), "
|
||||||
f"cycle count {cap.cycle_count}, 1 LSB = {lsb_nt:.2f} nT. "
|
f"cycle count {cap.cycle_count}, 1 LSB = {lsb_nt:.2f} nT. "
|
||||||
f"Sample grid is exact."
|
f"Drift {drift * 1e6:.0f} ppm — see the drift figure."
|
||||||
+ (f" {args.trim:g} s trimmed from each end." if trim_applied
|
+ (f" {args.trim:g} s trimmed from each end." if trim_applied
|
||||||
else ""),
|
else ""),
|
||||||
color=TEXT_SECONDARY, fontsize=10, ha="center")
|
color=TEXT_SECONDARY, fontsize=10, ha="center")
|
||||||
|
|
@ -893,19 +1277,22 @@ def main():
|
||||||
fig.savefig(out, facecolor=SURFACE)
|
fig.savefig(out, facecolor=SURFACE)
|
||||||
print(f"\n-> {out}")
|
print(f"\n-> {out}")
|
||||||
|
|
||||||
# The spectrogram is a second file rather than a fifth panel: it needs the
|
# Separate files rather than more panels. The spectrogram needs the width to
|
||||||
# width to resolve anything, and it answers a different question -- whether
|
# resolve anything; the other two answer questions the four panels do not --
|
||||||
# a feature was there throughout or only for part of the run.
|
# whether a feature lasted, how far the chip's clock moved, and which clock
|
||||||
spectrogram_path = out.rsplit(".", 1)[0] + "_spectrogram.png"
|
# each line is coherent on. Each degrades to a message rather than aborting
|
||||||
if spectrogram_path.endswith("_noise_spectrogram.png"):
|
# the run, since a short capture can legitimately support none of them.
|
||||||
spectrogram_path = spectrogram_path.replace("_noise_spectrogram",
|
for suffix, build in (("spectrogram",
|
||||||
"_spectrogram")
|
lambda p: make_spectrogram(cap, p, args.spectrogram,
|
||||||
|
args.overlap,
|
||||||
|
args.colormap,
|
||||||
|
args.max_freq)),
|
||||||
|
("drift", lambda p: make_drift(cap, p)),
|
||||||
|
("timebase", lambda p: make_time_base(cap, p))):
|
||||||
try:
|
try:
|
||||||
print(f"-> {make_spectrogram(cap, spectrogram_path, args.spectrogram,
|
print(f"-> {build(sibling_path(out, suffix))}")
|
||||||
args.overlap, args.colormap,
|
|
||||||
args.max_freq)}")
|
|
||||||
except ValueError as exc:
|
except ValueError as exc:
|
||||||
print(f"no spectrogram: {exc}")
|
print(f"no {suffix}: {exc}")
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|
|
||||||
113
compare.py
113
compare.py
|
|
@ -50,6 +50,7 @@ matplotlib.use("Agg")
|
||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
import calibrate
|
||||||
import capture
|
import capture
|
||||||
import characterize as ch
|
import characterize as ch
|
||||||
import rm3100
|
import rm3100
|
||||||
|
|
@ -260,16 +261,19 @@ def fit_rate_model(caps):
|
||||||
model that was never right: a supply that only scales the clock moves C and
|
model that was never right: a supply that only scales the clock moves C and
|
||||||
h together by one factor, leaving the *shape* of the model alone.
|
h together by one factor, leaving the *shape* of the model alone.
|
||||||
"""
|
"""
|
||||||
counts = np.array([float(c.cycle_count) for c in caps])
|
try:
|
||||||
if len(np.unique(counts)) < 2:
|
# The uncertainty on each period is how far the oscillator drifted
|
||||||
raise ValueError("need at least two distinct cycle counts to fit "
|
# during that capture; the worst of them bounds the fit.
|
||||||
"both the count rate and the per-axis overhead")
|
drift = max(calibrate.rate_stability(c) for c in caps)
|
||||||
periods = np.array([c.dt_true for c in caps]) / rm3100.AXES
|
fit = calibrate.fit_timing([c.cycle_count for c in caps],
|
||||||
slope, intercept = np.polyfit(counts, periods, 1)
|
[c.dt_true for c in caps],
|
||||||
if slope <= 0:
|
period_sd=drift or 1e-9)
|
||||||
raise ValueError("fitted count rate is not positive; the captures do "
|
return fit
|
||||||
"not follow the timing model")
|
except calibrate.CalibrationError as exc:
|
||||||
return 1.0 / float(slope), float(intercept)
|
# One fit, in calibrate.py, so sweep.py writing a calibration and this
|
||||||
|
# reporting one cannot drift apart. The exception type is translated
|
||||||
|
# because callers here already handle ValueError.
|
||||||
|
raise ValueError(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
def decimation_paths(fast, slow, k, band):
|
def decimation_paths(fast, slow, k, band):
|
||||||
|
|
@ -378,11 +382,13 @@ def locked_line_verdict(records):
|
||||||
# --------------------------------------------------------------------------
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
def print_overview(records, band, supplies):
|
def print_overview(records, band, supplies):
|
||||||
print(f"{'capture':<20} {'cc':>5} {'fs Hz':>9} {'|B| nT':>10} {'sd':>7} "
|
print(f"{'capture':<20} {'cc':>5} {'fs Hz':>9} {'drift':>7} {'|B| nT':>10} "
|
||||||
f"{'white':>7} {'ASD':>6} {'Allan':>6} | {'white':>7} {'ASD':>7}")
|
f"{'sd':>7} {'white':>7} {'ASD':>6} {'Allan':>6} | {'white':>7} "
|
||||||
|
f"{'ASD':>7}")
|
||||||
for label, _, s in records:
|
for label, _, s in records:
|
||||||
t = s["total"]
|
t = s["total"]
|
||||||
print(f"{label[:20]:<20} {s['cycle_count']:5d} {s['fs']:9.3f} "
|
print(f"{label[:20]:<20} {s['cycle_count']:5d} {s['fs']:9.3f} "
|
||||||
|
f"{calibrate.rate_stability(s['cap']) * 1e6:6.0f}p "
|
||||||
f"{s['field']:10,.0f} {t['sd']:7.1f} {t['white']:7.2f} "
|
f"{s['field']:10,.0f} {t['sd']:7.1f} {t['white']:7.2f} "
|
||||||
f"{t['asd']:6.2f} {s['allan'][0]:6.2f} | "
|
f"{t['asd']:6.2f} {s['allan'][0]:6.2f} | "
|
||||||
f"{t['white_ppm']:7.1f} {t['asd_ppm']:7.1f}")
|
f"{t['white_ppm']:7.1f} {t['asd_ppm']:7.1f}")
|
||||||
|
|
@ -397,6 +403,10 @@ def print_overview(records, band, supplies):
|
||||||
"quoted in nT at one supply.")
|
"quoted in nT at one supply.")
|
||||||
print(" white = sd(diff)/sqrt(2), which rejects drift; sd does not. "
|
print(" white = sd(diff)/sqrt(2), which rejects drift; sd does not. "
|
||||||
"Allan = best sigma by averaging.")
|
"Allan = best sigma by averaging.")
|
||||||
|
print(" drift is how far the oscillator moved across the run, in ppm -- the "
|
||||||
|
"error bar on that\n capture's rate, and on anything derived from it. "
|
||||||
|
"A tenth of a percent is ordinary\n for an LR relaxation oscillator. "
|
||||||
|
"It is a timing figure: nothing magnetic reaches it.")
|
||||||
if supplies:
|
if supplies:
|
||||||
rails = " ".join(f"{c} {v:g} V" for c, v in sorted(supplies.items()))
|
rails = " ".join(f"{c} {v:g} V" for c, v in sorted(supplies.items()))
|
||||||
print(f" rails: {rails}")
|
print(f" rails: {rails}")
|
||||||
|
|
@ -415,9 +425,16 @@ def print_rate_model(records):
|
||||||
if not fits:
|
if not fits:
|
||||||
return
|
return
|
||||||
print("\n=== timing model: period = 3 x (cycle_count / C + overhead) ===")
|
print("\n=== timing model: period = 3 x (cycle_count / C + overhead) ===")
|
||||||
for condition, (count_rate, overhead) in fits.items():
|
for condition, fit in fits.items():
|
||||||
print(f" {condition:<12} C {count_rate:10,.0f} counts/s "
|
print(f" {condition:<12} C {fit.counts_per_second:10,.0f} "
|
||||||
f"overhead {overhead * 1e6:6.2f} us")
|
f"+/- {fit.counts_per_second_sd:5,.0f} counts/s overhead "
|
||||||
|
f"{fit.axis_overhead_s * 1e6:6.2f} +/- "
|
||||||
|
f"{fit.axis_overhead_s_sd * 1e6:4.2f} us "
|
||||||
|
f"({fit.points} points, "
|
||||||
|
+ ("no residual: two points fit both terms exactly, so a bad "
|
||||||
|
"point cannot show up)"
|
||||||
|
if fit.points == 2 else
|
||||||
|
f"residual {fit.residual_ppm:.0f} ppm)"))
|
||||||
print(f" {'rm3100.py':<12} C {rm3100.COUNTS_PER_SECOND:10,.0f} counts/s "
|
print(f" {'rm3100.py':<12} C {rm3100.COUNTS_PER_SECOND:10,.0f} counts/s "
|
||||||
f"overhead {rm3100.AXIS_OVERHEAD_S * 1e6:6.2f} us (the model's own)")
|
f"overhead {rm3100.AXIS_OVERHEAD_S * 1e6:6.2f} us (the model's own)")
|
||||||
if len(fits) < 2:
|
if len(fits) < 2:
|
||||||
|
|
@ -427,8 +444,8 @@ def print_rate_model(records):
|
||||||
print(" changes its frequency scales C up and the overhead down by the "
|
print(" changes its frequency scales C up and the overhead down by the "
|
||||||
"same factor:")
|
"same factor:")
|
||||||
for (ca, fa), (cb, fb) in itertools.combinations(fits.items(), 2):
|
for (ca, fa), (cb, fb) in itertools.combinations(fits.items(), 2):
|
||||||
count_ratio = fb[0] / fa[0]
|
count_ratio = fb.counts_per_second / fa.counts_per_second
|
||||||
overhead_ratio = fb[1] / fa[1]
|
overhead_ratio = fb.axis_overhead_s / fa.axis_overhead_s
|
||||||
print(f" {ca} -> {cb} C x{count_ratio:.5f} "
|
print(f" {ca} -> {cb} C x{count_ratio:.5f} "
|
||||||
f"({(count_ratio - 1) * 100:+.2f}%) "
|
f"({(count_ratio - 1) * 100:+.2f}%) "
|
||||||
f"overhead x{overhead_ratio:.5f} "
|
f"overhead x{overhead_ratio:.5f} "
|
||||||
|
|
@ -513,18 +530,40 @@ CORRECTIONS = [("raw", "raw"),
|
||||||
("both", "x fs x volts")]
|
("both", "x fs x volts")]
|
||||||
|
|
||||||
|
|
||||||
def corrected_fields(s, supplies, reference_volts):
|
def correction_names(cal):
|
||||||
|
"""Row labels, naming what the rate row is actually referenced to.
|
||||||
|
|
||||||
|
Uncalibrated it is `1 + rate_error`, which measures the rate against
|
||||||
|
rm3100's *nominal* model -- and that model misfits by a cycle-count
|
||||||
|
dependent amount, so the row means something only inside a matched
|
||||||
|
cycle-count contrast. With a calibration it is the measured oscillator
|
||||||
|
against a stated reference, which means the same thing everywhere.
|
||||||
|
"""
|
||||||
|
renamed = {"rate": "x oscillator", "both": "x osc x volts"}
|
||||||
|
return [(key, renamed.get(key, name) if cal else name)
|
||||||
|
for key, name in CORRECTIONS]
|
||||||
|
|
||||||
|
|
||||||
|
def corrected_fields(s, supplies, reference_volts, cal=None):
|
||||||
"""|B| under each candidate correction, keyed by name; None if unavailable.
|
"""|B| under each candidate correction, keyed by name; None if unavailable.
|
||||||
|
|
||||||
rate the chip's own clock is measured every run, so a gain that tracks
|
rate the chip's own clock is measured every run, so a gain that tracks
|
||||||
the clock is correctable from the capture alone, with nothing
|
the clock is correctable from the capture alone. With a calibration
|
||||||
external to know.
|
this is the measured oscillator against a stated reference. Without
|
||||||
|
one it falls back to `1 + rate_error`, which references rm3100's
|
||||||
|
nominal model -- and that model's error is cycle-count dependent
|
||||||
|
(+0.84% at cc=100, -0.99% at cc=400 on one unit, same supply), so
|
||||||
|
the fallback is only meaningful inside a matched cycle-count
|
||||||
|
contrast, where it cancels.
|
||||||
supply a gain that is simply ratiometric with the rail needs the rail
|
supply a gain that is simply ratiometric with the rail needs the rail
|
||||||
measured, but nothing from the capture.
|
measured, but nothing from the capture.
|
||||||
both the two together, which is only right if they are independent
|
both the two together, which is only right if they are independent
|
||||||
mechanisms rather than one seen twice.
|
mechanisms rather than one seen twice.
|
||||||
"""
|
"""
|
||||||
raw = s["field"]
|
raw = s["field"]
|
||||||
|
if cal is not None:
|
||||||
|
by_rate = raw * calibrate.gain_factor(s["cap"], cal).value
|
||||||
|
else:
|
||||||
by_rate = raw * (1.0 + s["cap"].rate_error)
|
by_rate = raw * (1.0 + s["cap"].rate_error)
|
||||||
volts = supplies.get(condition_of(s["label"]))
|
volts = supplies.get(condition_of(s["label"]))
|
||||||
ratio = (volts / reference_volts) if volts else None
|
ratio = (volts / reference_volts) if volts else None
|
||||||
|
|
@ -534,7 +573,7 @@ def corrected_fields(s, supplies, reference_volts):
|
||||||
"both": by_rate * ratio if ratio else None}
|
"both": by_rate * ratio if ratio else None}
|
||||||
|
|
||||||
|
|
||||||
def print_contrasts(records, supplies):
|
def print_contrasts(records, supplies, cal=None):
|
||||||
"""Condition contrasts at matched cycle count -- the designed comparison."""
|
"""Condition contrasts at matched cycle count -- the designed comparison."""
|
||||||
conditions = conditions_in_order(records)
|
conditions = conditions_in_order(records)
|
||||||
if len(conditions) < 2:
|
if len(conditions) < 2:
|
||||||
|
|
@ -558,10 +597,10 @@ def print_contrasts(records, supplies):
|
||||||
if ca not in present or cb not in present:
|
if ca not in present or cb not in present:
|
||||||
continue
|
continue
|
||||||
a, b = present[ca], present[cb]
|
a, b = present[ca], present[cb]
|
||||||
left = corrected_fields(a, supplies, reference_volts)
|
left = corrected_fields(a, supplies, reference_volts, cal)
|
||||||
right = corrected_fields(b, supplies, reference_volts)
|
right = corrected_fields(b, supplies, reference_volts, cal)
|
||||||
print(f"\n cycle count {cc}: {ca} -> {cb}")
|
print(f"\n cycle count {cc}: {ca} -> {cb}")
|
||||||
for key, name in CORRECTIONS:
|
for key, name in correction_names(cal):
|
||||||
if left[key] is None or right[key] is None:
|
if left[key] is None or right[key] is None:
|
||||||
continue
|
continue
|
||||||
print(f" {name:<14} {left[key]:10,.1f} -> "
|
print(f" {name:<14} {left[key]:10,.1f} -> "
|
||||||
|
|
@ -708,7 +747,7 @@ def binned_asd(freqs, asd, bins=140):
|
||||||
return np.array(centres), np.array(values)
|
return np.array(centres), np.array(values)
|
||||||
|
|
||||||
|
|
||||||
def make_figure(records, band, supplies, path):
|
def make_figure(records, band, supplies, path, cal=None):
|
||||||
conditions = conditions_in_order(records)
|
conditions = conditions_in_order(records)
|
||||||
variants = sorted({variant_of(l) for l, _, _ in records})
|
variants = sorted({variant_of(l) for l, _, _ in records})
|
||||||
color = {c: CONDITION_COLORS[i % len(CONDITION_COLORS)]
|
color = {c: CONDITION_COLORS[i % len(CONDITION_COLORS)]
|
||||||
|
|
@ -736,7 +775,7 @@ def make_figure(records, band, supplies, path):
|
||||||
a.loglog(freqs, asd, color=color[condition_of(label)],
|
a.loglog(freqs, asd, color=color[condition_of(label)],
|
||||||
linestyle=dash[variant_of(label)], linewidth=1.4, alpha=0.9,
|
linestyle=dash[variant_of(label)], linewidth=1.4, alpha=0.9,
|
||||||
label=label)
|
label=label)
|
||||||
a.set_xlim(*ch.ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM)
|
a.set_xlim(*ch.FRACTIONAL_ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM)
|
||||||
ch.decade_ticks(a)
|
ch.decade_ticks(a)
|
||||||
title(a, "Spectral density of |B|, as a fraction of |B|")
|
title(a, "Spectral density of |B|, as a fraction of |B|")
|
||||||
a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz")
|
a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz")
|
||||||
|
|
@ -753,7 +792,7 @@ def make_figure(records, band, supplies, path):
|
||||||
title(a, "Fast capture, filtered and decimated")
|
title(a, "Fast capture, filtered and decimated")
|
||||||
a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz")
|
a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz")
|
||||||
if plotted:
|
if plotted:
|
||||||
a.set_xlim(*ch.ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM)
|
a.set_xlim(*ch.FRACTIONAL_ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM)
|
||||||
ch.decade_ticks(a)
|
ch.decade_ticks(a)
|
||||||
a.legend(frameon=False, fontsize=9, labelcolor=ch.TEXT_SECONDARY,
|
a.legend(frameon=False, fontsize=9, labelcolor=ch.TEXT_SECONDARY,
|
||||||
loc="upper right")
|
loc="upper right")
|
||||||
|
|
@ -779,7 +818,7 @@ def make_figure(records, band, supplies, path):
|
||||||
a.legend(frameon=False, fontsize=9, labelcolor=ch.TEXT_SECONDARY,
|
a.legend(frameon=False, fontsize=9, labelcolor=ch.TEXT_SECONDARY,
|
||||||
loc="upper right")
|
loc="upper right")
|
||||||
|
|
||||||
_plot_corrections(axs[1, 1], records, supplies, color, marker)
|
_plot_corrections(axs[1, 1], records, supplies, color, marker, cal)
|
||||||
|
|
||||||
fig.suptitle("RM3100 capture comparison", color=ch.TEXT_PRIMARY,
|
fig.suptitle("RM3100 capture comparison", color=ch.TEXT_PRIMARY,
|
||||||
fontsize=15, fontweight="bold", y=0.985)
|
fontsize=15, fontweight="bold", y=0.985)
|
||||||
|
|
@ -826,14 +865,14 @@ def _plot_decimation(ax, records, band, color, dash):
|
||||||
return plotted
|
return plotted
|
||||||
|
|
||||||
|
|
||||||
def _plot_corrections(ax, records, supplies, color, marker):
|
def _plot_corrections(ax, records, supplies, color, marker, cal=None):
|
||||||
"""|B| under each correction, so the spread each one leaves is visible."""
|
"""|B| under each correction, so the spread each one leaves is visible."""
|
||||||
reference_volts = min(supplies.values()) if supplies else None
|
reference_volts = min(supplies.values()) if supplies else None
|
||||||
for label, _, s in records:
|
for label, _, s in records:
|
||||||
s["label"] = label
|
s["label"] = label
|
||||||
corrected = [(label, corrected_fields(s, supplies, reference_volts))
|
corrected = [(label, corrected_fields(s, supplies, reference_volts, cal))
|
||||||
for label, _, s in records]
|
for label, _, s in records]
|
||||||
rows = [(key, name) for key, name in CORRECTIONS
|
rows = [(key, name) for key, name in correction_names(cal)
|
||||||
if all(c[key] is not None for _, c in corrected)]
|
if all(c[key] is not None for _, c in corrected)]
|
||||||
|
|
||||||
for row, (key, name) in enumerate(rows):
|
for row, (key, name) in enumerate(rows):
|
||||||
|
|
@ -891,10 +930,18 @@ def main():
|
||||||
default=[], metavar="CONDITION=VOLTS",
|
default=[], metavar="CONDITION=VOLTS",
|
||||||
help="rail voltage for a condition, enabling the "
|
help="rail voltage for a condition, enabling the "
|
||||||
"ratiometric correction; repeatable")
|
"ratiometric correction; repeatable")
|
||||||
|
ap.add_argument("--calibration", default=None, metavar="JSON",
|
||||||
|
help="reference the rate correction to a measured "
|
||||||
|
"calibration instead of rm3100's nominal timing model")
|
||||||
ap.add_argument("-o", "--output", default=None, metavar="PNG",
|
ap.add_argument("-o", "--output", default=None, metavar="PNG",
|
||||||
help="write a comparison figure here")
|
help="write a comparison figure here")
|
||||||
args = ap.parse_args()
|
args = ap.parse_args()
|
||||||
supplies = dict(args.supply)
|
supplies = dict(args.supply)
|
||||||
|
try:
|
||||||
|
cal = (calibrate.load_calibration(args.calibration)
|
||||||
|
if args.calibration else None)
|
||||||
|
except (OSError, calibrate.CalibrationError) as exc:
|
||||||
|
sys.exit(str(exc))
|
||||||
|
|
||||||
loaded = []
|
loaded = []
|
||||||
for argument in args.csv:
|
for argument in args.csv:
|
||||||
|
|
@ -937,11 +984,11 @@ def main():
|
||||||
print_rate_model(records)
|
print_rate_model(records)
|
||||||
print_locked_lines(records)
|
print_locked_lines(records)
|
||||||
print_decimation(records, band)
|
print_decimation(records, band)
|
||||||
print_contrasts(records, supplies)
|
print_contrasts(records, supplies, cal)
|
||||||
print_pairwise(records)
|
print_pairwise(records)
|
||||||
|
|
||||||
if args.output:
|
if args.output:
|
||||||
print(f"\n-> {make_figure(records, band, supplies, args.output)}")
|
print(f"\n-> {make_figure(records, band, supplies, args.output, cal)}")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
17
plot.py
17
plot.py
|
|
@ -18,6 +18,7 @@ matplotlib.use("Agg")
|
||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
import calibrate
|
||||||
import capture
|
import capture
|
||||||
|
|
||||||
# Light-mode design tokens.
|
# Light-mode design tokens.
|
||||||
|
|
@ -65,8 +66,11 @@ def main():
|
||||||
args = ap.parse_args()
|
args = ap.parse_args()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
cap = capture.load(args.csv)
|
# Either format: a raw capture, or one already converted to nT by
|
||||||
except (OSError, capture.CaptureError) as exc:
|
# calibrate.py. Which it is, is a property of the file.
|
||||||
|
cap = calibrate.load_any(args.csv)
|
||||||
|
except (OSError, capture.CaptureError,
|
||||||
|
calibrate.CalibrationError) as exc:
|
||||||
sys.exit(str(exc))
|
sys.exit(str(exc))
|
||||||
|
|
||||||
# Drift-corrected time: the nominal grid would be ~6% off real seconds.
|
# Drift-corrected time: the nominal grid would be ~6% off real seconds.
|
||||||
|
|
@ -119,10 +123,15 @@ def main():
|
||||||
if window > 1 else "")
|
if window > 1 else "")
|
||||||
fig.suptitle("RM3100 magnetometer capture", color=TEXT_PRIMARY,
|
fig.suptitle("RM3100 magnetometer capture", color=TEXT_PRIMARY,
|
||||||
fontsize=15, fontweight="bold", x=0.5, y=0.985)
|
fontsize=15, fontweight="bold", x=0.5, y=0.985)
|
||||||
|
# A calibrated file has no nominal rate to be wrong against -- it has a
|
||||||
|
# gain factor instead, which is the thing worth stating about it.
|
||||||
|
provenance = (cap.describe() if hasattr(cap, "describe") else
|
||||||
|
f"nominal {cap.nominal_rate_hz:g} Hz, "
|
||||||
|
f"{cap.rate_error * 100:+.1f}%")
|
||||||
fig.text(0.5, 0.955,
|
fig.text(0.5, 0.955,
|
||||||
f"{len(t):,} samples over {duration:.1f} s at {rate:.2f} Hz "
|
f"{len(t):,} samples over {duration:.1f} s at {rate:.2f} Hz "
|
||||||
f"(nominal {cap.nominal_rate_hz:g} Hz, {cap.rate_error * 100:+.1f}%), "
|
f"({provenance}), cycle count {cap.cycle_count} "
|
||||||
f"cycle count {cap.cycle_count} (1 LSB = {cap.lsb_nt:.1f} nT)"
|
f"(1 LSB = {cap.lsb_nt:.1f} nT)"
|
||||||
f"{smooth_note}. Panels have independent y-scales.",
|
f"{smooth_note}. Panels have independent y-scales.",
|
||||||
color=TEXT_SECONDARY, fontsize=10, ha="center")
|
color=TEXT_SECONDARY, fontsize=10, ha="center")
|
||||||
|
|
||||||
|
|
|
||||||
16
rm3100.py
16
rm3100.py
|
|
@ -108,8 +108,22 @@ def tesla_per_count(cycle_count):
|
||||||
# the observed rates to better than 1% when the divisor is held at spec. It is
|
# the observed rates to better than 1% when the divisor is held at spec. It is
|
||||||
# why the naive "rate x cycle_count" constant is not constant, drifting from
|
# why the naive "rate x cycle_count" constant is not constant, drifting from
|
||||||
# 84,429 at cycle count 100 to 89,191 at 1200.
|
# 84,429 at cycle count 100 to 89,191 at 1200.
|
||||||
|
#
|
||||||
|
# **This pair is a nominal model, not a description of any particular part.**
|
||||||
|
# Solving it freely against one unit's two cycle counts gave 88,546 counts/s
|
||||||
|
# with a 40.6 us overhead at 3.006 V, and 92,889 with 38.1 us at 3.353 V -- so
|
||||||
|
# both terms are per-unit *and* move with the supply, because the oscillator
|
||||||
|
# does. Holding the divisor at spec and solving for the overhead alone cannot
|
||||||
|
# absorb it: that gives 58.9 us at cycle count 100 and 113.6 us at 400 on the
|
||||||
|
# same data, fitting neither.
|
||||||
|
#
|
||||||
|
# The consequence to know about is that the error is *cycle-count dependent*.
|
||||||
|
# On that unit the model was +0.84% fast at cycle count 100 and -0.99% slow at
|
||||||
|
# 400 -- same supply, opposite signs -- so `Capture.rate_error` measured against
|
||||||
|
# it is not a gain indicator. calibrate.py holds the per-unit pair, and
|
||||||
|
# `sweep.py --calibration-out` measures one.
|
||||||
COUNTS_PER_SECOND = 90000.0 # 180 kHz (Table 3-1) / 2 bias directions
|
COUNTS_PER_SECOND = 90000.0 # 180 kHz (Table 3-1) / 2 bias directions
|
||||||
AXIS_OVERHEAD_S = 68.7e-6
|
AXIS_OVERHEAD_S = 68.7e-6 # nominal; measured parts run 38-41 us
|
||||||
AXES = 3
|
AXES = 3
|
||||||
|
|
||||||
# Rates predicted from the model are good to roughly this much on a given unit,
|
# Rates predicted from the model are good to roughly this much on a given unit,
|
||||||
|
|
|
||||||
71
sweep.py
71
sweep.py
|
|
@ -19,8 +19,10 @@ import argparse
|
||||||
import re
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
import calibrate
|
||||||
import ch347
|
import ch347
|
||||||
import logger
|
import logger
|
||||||
import rm3100
|
import rm3100
|
||||||
|
|
@ -30,6 +32,12 @@ CALIBRATED = re.compile(r"Calibrated period [\d.]+ ms = ([\d.]+) Hz")
|
||||||
MISSED = re.compile(r"WARNING: ([\d,]+) measurement\(s\) were lost")
|
MISSED = re.compile(r"WARNING: ([\d,]+) measurement\(s\) were lost")
|
||||||
AMBIGUOUS = re.compile(r"WARNING: ([\d,]+) gap\(s\) could not be counted")
|
AMBIGUOUS = re.compile(r"WARNING: ([\d,]+) gap\(s\) could not be counted")
|
||||||
|
|
||||||
|
# Each sweep point is a short capture, so the oscillator barely drifts within
|
||||||
|
# one -- but the points are minutes apart and the chip warms through the sweep.
|
||||||
|
# This is the fractional period uncertainty that implies, and it is what the
|
||||||
|
# timing fit's error bars are built on.
|
||||||
|
PERIOD_SD = 500e-6
|
||||||
|
|
||||||
CALIBRATE_SECONDS = 5.0
|
CALIBRATE_SECONDS = 5.0
|
||||||
RECORD_SECONDS = 5.0
|
RECORD_SECONDS = 5.0
|
||||||
|
|
||||||
|
|
@ -72,6 +80,57 @@ def measure(rate, bus_speed, tmrc, output, record_seconds):
|
||||||
return cc, None, "failed"
|
return cc, None, "failed"
|
||||||
|
|
||||||
|
|
||||||
|
def write_calibration(measured, path, note):
|
||||||
|
"""Fit the timing model across the sweep and save it as a calibration.
|
||||||
|
|
||||||
|
The sweep already measures what a calibration is made of -- a rate at each
|
||||||
|
of several cycle counts -- so fitting it here is the difference between a
|
||||||
|
per-unit reference that was measured and one that was assumed. The reference
|
||||||
|
oscillator is set to the fitted value, which makes this sweep the definition
|
||||||
|
of "uncorrected" and every later capture correct relative to it.
|
||||||
|
"""
|
||||||
|
if len({cc for cc, _ in measured}) < 2:
|
||||||
|
print(f"\nnot writing {path}: the sweep produced "
|
||||||
|
f"{len({cc for cc, _ in measured})} distinct cycle count(s), and "
|
||||||
|
"fitting both\nthe count rate and the overhead needs at least "
|
||||||
|
"two.", file=sys.stderr)
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
fit = calibrate.fit_timing([cc for cc, _ in measured],
|
||||||
|
[period for _, period in measured],
|
||||||
|
period_sd=PERIOD_SD)
|
||||||
|
except calibrate.CalibrationError as exc:
|
||||||
|
print(f"\nnot writing {path}: {exc}", file=sys.stderr)
|
||||||
|
return None
|
||||||
|
|
||||||
|
cal = calibrate.calibration(
|
||||||
|
counts_per_second=fit.counts_per_second,
|
||||||
|
axis_overhead_s=fit.axis_overhead_s,
|
||||||
|
reference_oscillator_hz=fit.counts_per_second,
|
||||||
|
reference_cycle_count=min(cc for cc, _ in measured),
|
||||||
|
gain_exponent=1.0, gain_offset_counts=0.0,
|
||||||
|
counts_per_second_sd=fit.counts_per_second_sd,
|
||||||
|
axis_overhead_s_sd=fit.axis_overhead_s_sd,
|
||||||
|
note=note or f"sweep of {len(measured)} points, "
|
||||||
|
f"cycle counts {min(cc for cc, _ in measured)}"
|
||||||
|
f"-{max(cc for cc, _ in measured)}",
|
||||||
|
created=time.strftime("%Y-%m-%dT%H:%M:%S"))
|
||||||
|
calibrate.save_calibration(path, cal)
|
||||||
|
print(f"\ncalibration -> {path}")
|
||||||
|
print(f" counts/s {fit.counts_per_second:,.0f} +/- "
|
||||||
|
f"{fit.counts_per_second_sd:,.0f} overhead "
|
||||||
|
f"{fit.axis_overhead_s * 1e6:.2f} +/- "
|
||||||
|
f"{fit.axis_overhead_s_sd * 1e6:.2f} us "
|
||||||
|
f"(the model's own: {rm3100.COUNTS_PER_SECOND:,.0f} and "
|
||||||
|
f"{rm3100.AXIS_OVERHEAD_S * 1e6:.1f} us)")
|
||||||
|
print(f" fit residual {fit.residual_ppm:.0f} ppm over {fit.points} points")
|
||||||
|
print(" gain_exponent is 1.0, which the oscillator-clocked measurement "
|
||||||
|
"interval predicts,\n and gain_offset_counts is 0.0. Neither is "
|
||||||
|
"measured by a rate sweep -- both need\n a field that does not move "
|
||||||
|
"between cycle counts. See NOTES.md.")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
ap = argparse.ArgumentParser(description=__doc__,
|
ap = argparse.ArgumentParser(description=__doc__,
|
||||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
|
@ -88,6 +147,12 @@ def main():
|
||||||
ap.add_argument("--duration", type=float, default=RECORD_SECONDS,
|
ap.add_argument("--duration", type=float, default=RECORD_SECONDS,
|
||||||
help="seconds to record at each point, after calibration "
|
help="seconds to record at each point, after calibration "
|
||||||
"(default: %(default)s)")
|
"(default: %(default)s)")
|
||||||
|
ap.add_argument("--calibration-out", default=None, metavar="JSON",
|
||||||
|
help="fit the timing model across the sweep and write it "
|
||||||
|
"as a calibration for calibrate.py")
|
||||||
|
ap.add_argument("--note", default=None,
|
||||||
|
help="note recorded in the calibration, e.g. the unit and "
|
||||||
|
"supply it was measured at")
|
||||||
ap.add_argument("--output", default="/tmp/sweep_point.csv",
|
ap.add_argument("--output", default="/tmp/sweep_point.csv",
|
||||||
help="scratch capture path, overwritten each point")
|
help="scratch capture path, overwritten each point")
|
||||||
args = ap.parse_args()
|
args = ap.parse_args()
|
||||||
|
|
@ -107,9 +172,12 @@ def main():
|
||||||
print(row(name for name, _ in COLUMNS))
|
print(row(name for name, _ in COLUMNS))
|
||||||
print(row("-" * width for _, width in COLUMNS))
|
print(row("-" * width for _, width in COLUMNS))
|
||||||
|
|
||||||
|
measured = []
|
||||||
for rate in rates:
|
for rate in rates:
|
||||||
cc, hz, note = measure(rate, args.bus_speed, args.tmrc, args.output,
|
cc, hz, note = measure(rate, args.bus_speed, args.tmrc, args.output,
|
||||||
args.duration)
|
args.duration)
|
||||||
|
if hz:
|
||||||
|
measured.append((cc, 1.0 / hz))
|
||||||
# One count is one LSB, so nT/LSB is the quantisation step directly.
|
# One count is one LSB, so nT/LSB is the quantisation step directly.
|
||||||
nt_per_lsb = rm3100.tesla_per_count(cc) * rm3100.NT_PER_TESLA
|
nt_per_lsb = rm3100.tesla_per_count(cc) * rm3100.NT_PER_TESLA
|
||||||
noise = rm3100.expected_noise_nt(cc)
|
noise = rm3100.expected_noise_nt(cc)
|
||||||
|
|
@ -140,6 +208,9 @@ def main():
|
||||||
"bus use is irreducible I2C traffic against the period -- not occupancy,\n"
|
"bus use is irreducible I2C traffic against the period -- not occupancy,\n"
|
||||||
"which approaches 100% because the loop polls continuously for DRDY.")
|
"which approaches 100% because the loop polls continuously for DRDY.")
|
||||||
|
|
||||||
|
if args.calibration_out:
|
||||||
|
write_calibration(measured, args.calibration_out, args.note)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
main()
|
main()
|
||||||
|
|
|
||||||
|
|
@ -22,13 +22,19 @@ def write_capture(path, rows=200, dt=DT, nominal_hz=282.0, header=None,
|
||||||
flags=None, drop_header_key=None, extra_lines=(),
|
flags=None, drop_header_key=None, extra_lines=(),
|
||||||
index_from=0, index_step=1, start_time=1_700_000_000.0,
|
index_from=0, index_step=1, start_time=1_700_000_000.0,
|
||||||
amplitude=1000.0, seed=0, cycle_count=CYCLE_COUNT,
|
amplitude=1000.0, seed=0, cycle_count=CYCLE_COUNT,
|
||||||
counts=None):
|
counts=None, times=None):
|
||||||
"""Write a synthetic capture and return its path.
|
"""Write a synthetic capture and return its path.
|
||||||
|
|
||||||
`counts` overrides the generated signal with an (rows, 3) array of raw
|
`counts` overrides the generated signal with an (rows, 3) array of raw
|
||||||
counts, which is how a test plants an exact answer. `cycle_count` moves the
|
counts, which is how a test plants an exact answer. `cycle_count` moves the
|
||||||
header's gain and is what makes a decimation pair possible: two captures
|
header's gain and is what makes a decimation pair possible: two captures
|
||||||
whose cycle counts differ by an integer factor.
|
whose cycle counts differ by an integer factor.
|
||||||
|
|
||||||
|
`times` overrides the timestamps with absolute unix seconds, one per row.
|
||||||
|
Without it the grid is exactly uniform, which is the right default but makes
|
||||||
|
a whole class of question untestable: the chip's oscillator drifts through a
|
||||||
|
real run, and the analysis that measures the drift needs a capture where the
|
||||||
|
answer was planted. `drifting_times` below builds the usual case.
|
||||||
"""
|
"""
|
||||||
meta = {
|
meta = {
|
||||||
"rm3100_capture": 1,
|
"rm3100_capture": 1,
|
||||||
|
|
@ -62,11 +68,32 @@ def write_capture(path, rows=200, dt=DT, nominal_hz=282.0, header=None,
|
||||||
x = int(amplitude + rng.normal(0, 3))
|
x = int(amplitude + rng.normal(0, 3))
|
||||||
y = int(2 * amplitude + rng.normal(0, 3))
|
y = int(2 * amplitude + rng.normal(0, 3))
|
||||||
z = int(-amplitude + rng.normal(0, 3))
|
z = int(-amplitude + rng.normal(0, 3))
|
||||||
lines.append(f"{index},{start_time + index * dt:.6f},{x},{y},{z},{warning}")
|
stamp = start_time + index * dt if times is None else times[i]
|
||||||
|
lines.append(f"{index},{stamp:.6f},{x},{y},{z},{warning}")
|
||||||
path.write_text("\n".join(lines) + "\n")
|
path.write_text("\n".join(lines) + "\n")
|
||||||
return str(path)
|
return str(path)
|
||||||
|
|
||||||
|
|
||||||
|
def drifting_times(rows, dt, ppm_per_second=0.0, jitter_s=0.0, seed=0,
|
||||||
|
start_time=1_700_000_000.0):
|
||||||
|
"""Timestamps for a capture whose sample period ramps linearly.
|
||||||
|
|
||||||
|
The period at sample k is dt * (1 + ppm_per_second * 1e-6 * t), so the times
|
||||||
|
are the integral of that -- quadratic in k, which is what a warming
|
||||||
|
oscillator actually produces and what a straight-line fit of time against
|
||||||
|
index cannot absorb.
|
||||||
|
|
||||||
|
`jitter_s` adds independent noise to each timestamp without moving the
|
||||||
|
underlying grid, standing in for host scheduling: it is what the drift has
|
||||||
|
to be measured through, and it must not be mistaken for drift.
|
||||||
|
"""
|
||||||
|
k = np.arange(rows)
|
||||||
|
t = k * dt + 0.5 * ppm_per_second * 1e-6 * dt * k * (k - 1) * dt
|
||||||
|
if jitter_s:
|
||||||
|
t = t + np.random.default_rng(seed).normal(0, jitter_s, rows)
|
||||||
|
return start_time + t
|
||||||
|
|
||||||
|
|
||||||
def field_counts(rows, mean_nt, cycle_count, noise_nt=0.0, seed=0, tones=()):
|
def field_counts(rows, mean_nt, cycle_count, noise_nt=0.0, seed=0, tones=()):
|
||||||
"""Raw counts for a field of a given mean, noise and planted tones.
|
"""Raw counts for a field of a given mean, noise and planted tones.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -5,13 +5,17 @@ so a normalisation slip -- the easy mistake in Welch and Allan code, and an
|
||||||
invisible one on real data -- shows up as a factor rather than a wobble.
|
invisible one on real data -- shows up as a factor rather than a wobble.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
import capture
|
||||||
import characterize
|
import characterize
|
||||||
import compare
|
import compare
|
||||||
import logger
|
import logger
|
||||||
import plot
|
import plot
|
||||||
|
from synthetic import drifting_times, field_counts, write_capture
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------
|
# --------------------------------------------------------------------------
|
||||||
|
|
@ -219,3 +223,168 @@ def test_axis_ratio_spread_abstains_with_too_few_usable_axes():
|
||||||
a = _record([10_000, 3, 5])
|
a = _record([10_000, 3, 5])
|
||||||
b = _record([10_600, -8, 2])
|
b = _record([10_600, -8, 2])
|
||||||
assert compare.axis_ratio_spread(a, b, b["mean"] / a["mean"]) is None
|
assert compare.axis_ratio_spread(a, b, b["mean"] / a["mean"]) is None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# coherent_amplitude, host_time_base, and the time-base test they support
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
TB_DT = 1.0 / 250.0
|
||||||
|
TB_ROWS = 25_500
|
||||||
|
TB_FIELD = (-40000.0, 16000.0, 12000.0)
|
||||||
|
|
||||||
|
|
||||||
|
def _drifted(path, ppm_per_second=0.0, jitter_s=0.0, tones=(), noise_nt=5.0,
|
||||||
|
rows=TB_ROWS, seed=0):
|
||||||
|
counts = field_counts(rows, TB_FIELD, 100, noise_nt, seed, tones)
|
||||||
|
return capture.load(write_capture(
|
||||||
|
path, rows=rows, dt=TB_DT, counts=counts,
|
||||||
|
times=drifting_times(rows, TB_DT, ppm_per_second, jitter_s, seed)))
|
||||||
|
|
||||||
|
|
||||||
|
def test_coherent_amplitude_recovers_a_planted_tone():
|
||||||
|
"""2*|mean(v exp(-i w t))| is the tone's peak amplitude, not its rms."""
|
||||||
|
fs, n = 250.0, 20_000
|
||||||
|
t = np.arange(n) / fs
|
||||||
|
v = 7.0 * np.cos(2 * np.pi * 31.0 * t + 0.4)
|
||||||
|
assert characterize.coherent_amplitude(v, t, 31.0)[0] == pytest.approx(7.0,
|
||||||
|
rel=1e-3)
|
||||||
|
|
||||||
|
|
||||||
|
def test_coherent_amplitude_is_near_zero_away_from_the_tone():
|
||||||
|
fs, n = 250.0, 20_000
|
||||||
|
t = np.arange(n) / fs
|
||||||
|
v = 7.0 * np.cos(2 * np.pi * 31.0 * t)
|
||||||
|
assert characterize.coherent_amplitude(v, t, 40.0)[0] < 0.05
|
||||||
|
|
||||||
|
|
||||||
|
def test_coherent_amplitude_accepts_many_frequencies_at_once():
|
||||||
|
t = np.arange(20_000) / 250.0
|
||||||
|
out = characterize.coherent_amplitude(np.cos(2 * np.pi * 31.0 * t), t,
|
||||||
|
[20.0, 31.0, 40.0])
|
||||||
|
assert len(out) == 3
|
||||||
|
assert out[1] > 10 * max(out[0], out[2])
|
||||||
|
|
||||||
|
|
||||||
|
def test_coherent_amplitude_ignores_a_dc_offset():
|
||||||
|
t = np.arange(20_000) / 250.0
|
||||||
|
v = 3.0 * np.cos(2 * np.pi * 31.0 * t)
|
||||||
|
assert characterize.coherent_amplitude(v + 5000.0, t, 31.0)[0] == \
|
||||||
|
pytest.approx(characterize.coherent_amplitude(v, t, 31.0)[0], rel=1e-6)
|
||||||
|
|
||||||
|
|
||||||
|
def test_coherent_amplitude_reads_an_unevenly_sampled_tone():
|
||||||
|
"""The reason this exists rather than a call to welch_asd.
|
||||||
|
|
||||||
|
Samples taken at irregular times still carry the tone; a spectrum, which can
|
||||||
|
only assume they were evenly spaced, puts the energy somewhere else.
|
||||||
|
"""
|
||||||
|
rng = np.random.default_rng(0)
|
||||||
|
t = np.sort(rng.uniform(0, 80.0, 20_000))
|
||||||
|
v = 7.0 * np.cos(2 * np.pi * 31.0 * t)
|
||||||
|
assert characterize.coherent_amplitude(v, t, 31.0)[0] == pytest.approx(7.0,
|
||||||
|
rel=0.02)
|
||||||
|
# The same samples read as if they were uniform: the tone is gone.
|
||||||
|
uniform = np.arange(len(t)) * (t[-1] / len(t))
|
||||||
|
assert characterize.coherent_amplitude(v, uniform, 31.0)[0] < 1.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_host_time_base_at_degree_one_is_the_uniform_grid(tmp_path):
|
||||||
|
"""Degree 1 is the grid already in use, so it is the family's null case."""
|
||||||
|
cap = _drifted(tmp_path / "d.csv", ppm_per_second=8.0)
|
||||||
|
linear = characterize.host_time_base(cap, degree=1)
|
||||||
|
grid = (cap.sample_index - cap.sample_index[0]) * cap.dt_true
|
||||||
|
assert np.allclose(linear, grid, atol=1e-9)
|
||||||
|
|
||||||
|
|
||||||
|
def test_host_time_base_follows_a_planted_drift(tmp_path):
|
||||||
|
"""Degree 3 tracks the ramp the uniform grid cannot absorb."""
|
||||||
|
cap = _drifted(tmp_path / "d.csv", ppm_per_second=8.0)
|
||||||
|
truth = cap.system_time - cap.system_time[0]
|
||||||
|
grid = (cap.sample_index - cap.sample_index[0]) * cap.dt_true
|
||||||
|
fitted = characterize.host_time_base(cap)
|
||||||
|
assert np.abs(fitted - truth).max() < np.abs(grid - truth).max() / 100
|
||||||
|
|
||||||
|
|
||||||
|
def test_host_time_base_rejects_read_jitter(tmp_path):
|
||||||
|
"""Scheduling noise says nothing about when the chip sampled, so it goes."""
|
||||||
|
cap = _drifted(tmp_path / "j.csv", jitter_s=2e-3)
|
||||||
|
fitted = characterize.host_time_base(cap)
|
||||||
|
raw = cap.system_time - cap.system_time[0]
|
||||||
|
assert (fitted - raw).std() == pytest.approx(2e-3, rel=0.1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_host_time_base_degrades_on_a_capture_shorter_than_its_degree(tmp_path):
|
||||||
|
cap = _drifted(tmp_path / "s.csv", rows=200)
|
||||||
|
assert len(characterize.host_time_base(cap, degree=500)) == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_wall_clock_tone_prefers_the_host_time_base(tmp_path):
|
||||||
|
"""The discriminator's claim, on data where the answer is planted.
|
||||||
|
|
||||||
|
A tone at a fixed frequency in wall time is smeared by a drifting grid, so
|
||||||
|
placing the samples by the host clock recovers amplitude the index grid lost.
|
||||||
|
"""
|
||||||
|
cap = _drifted(tmp_path / "w.csv", ppm_per_second=40.0, noise_nt=1.0)
|
||||||
|
# 31 Hz in wall time, written against the true sample times.
|
||||||
|
t = cap.system_time - cap.system_time[0]
|
||||||
|
v = 40.0 * np.cos(2 * np.pi * 31.0 * t)
|
||||||
|
grid = (cap.sample_index - cap.sample_index[0]) * cap.dt_true
|
||||||
|
on_grid = characterize.coherent_amplitude(v, grid, 31.0)[0]
|
||||||
|
on_host = characterize.coherent_amplitude(v, characterize.host_time_base(cap),
|
||||||
|
31.0)[0]
|
||||||
|
assert on_host > 1.5 * on_grid
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_sample_locked_tone_prefers_the_index_grid(tmp_path):
|
||||||
|
"""The control. A tone locked to the sampling is wrecked by the host clock."""
|
||||||
|
cap = _drifted(tmp_path / "l.csv", ppm_per_second=40.0, noise_nt=1.0)
|
||||||
|
index = cap.sample_index - cap.sample_index[0]
|
||||||
|
v = 40.0 * np.cos(2 * np.pi * 0.124 * index) # a fixed fraction of fs
|
||||||
|
grid = index * cap.dt_true
|
||||||
|
on_grid = characterize.coherent_amplitude(v, grid, 0.124 / cap.dt_true)[0]
|
||||||
|
on_host = characterize.coherent_amplitude(v, characterize.host_time_base(cap),
|
||||||
|
0.124 / cap.dt_true)[0]
|
||||||
|
assert on_host < 0.75 * on_grid
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# sibling_path and the figures
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("path, suffix, expected", [
|
||||||
|
("run_noise.png", "drift", "run_drift.png"),
|
||||||
|
("run.png", "drift", "run_drift.png"),
|
||||||
|
("/a/b/run_noise.png", "timebase", "/a/b/run_timebase.png"),
|
||||||
|
("run_noise.pdf", "spectrogram", "run_spectrogram.png"),
|
||||||
|
("noise_run_noise.png", "drift", "noise_run_drift.png"),
|
||||||
|
("a.b.c_noise.png", "drift", "a.b.c_drift.png"),
|
||||||
|
])
|
||||||
|
def test_sibling_path_names_a_companion_figure(path, suffix, expected):
|
||||||
|
assert characterize.sibling_path(path, suffix) == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_make_drift_writes_a_figure(tmp_path):
|
||||||
|
cap = _drifted(tmp_path / "d.csv", ppm_per_second=8.0)
|
||||||
|
out = characterize.make_drift(cap, str(tmp_path / "d_drift.png"))
|
||||||
|
assert Path(out).stat().st_size > 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_make_drift_declines_a_capture_too_short_to_window(tmp_path):
|
||||||
|
"""A short capture is a reason to say so, not to abort the whole run."""
|
||||||
|
cap = _drifted(tmp_path / "s.csv", rows=2_000)
|
||||||
|
with pytest.raises(ValueError, match="too few|too short"):
|
||||||
|
characterize.make_drift(cap, str(tmp_path / "s_drift.png"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_make_time_base_writes_a_figure(tmp_path):
|
||||||
|
cap = _drifted(tmp_path / "t.csv", ppm_per_second=8.0,
|
||||||
|
tones=[(0, 0.25, 60.0)])
|
||||||
|
out = characterize.make_time_base(cap, str(tmp_path / "t_timebase.png"))
|
||||||
|
assert Path(out).stat().st_size > 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_make_time_base_declines_a_constant_field(tmp_path):
|
||||||
|
cap = _drifted(tmp_path / "c.csv", noise_nt=0.0)
|
||||||
|
with pytest.raises(ValueError, match="constant|no lines"):
|
||||||
|
characterize.make_time_base(cap, str(tmp_path / "c_timebase.png"))
|
||||||
|
|
|
||||||
708
tests/test_calibrate.py
Normal file
708
tests/test_calibrate.py
Normal file
|
|
@ -0,0 +1,708 @@
|
||||||
|
"""Calibration: the timing fit, the gain factor, and the file it is carried in.
|
||||||
|
|
||||||
|
Everything is checked against a planted answer. A calibration is a claim about a
|
||||||
|
particular part, so the tests that matter are the ones showing a value goes in
|
||||||
|
and the same value comes back out -- and that a file which cannot be fully
|
||||||
|
understood is refused rather than half-read into a plausible-looking correction.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import calibrate
|
||||||
|
import capture
|
||||||
|
import rm3100
|
||||||
|
from synthetic import drifting_times, field_counts, write_capture
|
||||||
|
|
||||||
|
FIELD = (-40000.0, 16000.0, 12000.0)
|
||||||
|
|
||||||
|
|
||||||
|
def planted_period(cycle_count, counts_per_second, overhead):
|
||||||
|
return rm3100.AXES * (cycle_count / counts_per_second + overhead)
|
||||||
|
|
||||||
|
|
||||||
|
def make(path, cycle_count=100, counts_per_second=88546.0, overhead=40.61e-6,
|
||||||
|
rows=2000, mean=FIELD, noise_nt=5.0, seed=0):
|
||||||
|
counts = field_counts(rows, mean, cycle_count, noise_nt, seed)
|
||||||
|
return capture.load(write_capture(
|
||||||
|
path, rows=rows, cycle_count=cycle_count, counts=counts,
|
||||||
|
dt=planted_period(cycle_count, counts_per_second, overhead)))
|
||||||
|
|
||||||
|
|
||||||
|
def written(tmp_path, **fields):
|
||||||
|
body = {"rm3100_calibration": calibrate.CALIBRATION_VERSION,
|
||||||
|
"counts_per_second": 88546.0, "axis_overhead_s": 40.61e-6}
|
||||||
|
body.update(fields)
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
path.write_text(json.dumps(body))
|
||||||
|
return str(path)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# fit_timing
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_fit_timing_recovers_a_planted_pair_exactly():
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
ccs = [100, 400]
|
||||||
|
periods = [planted_period(cc, counts_per_second, overhead) for cc in ccs]
|
||||||
|
fit = calibrate.fit_timing(ccs, periods, period_sd=1e-4)
|
||||||
|
assert fit.counts_per_second == pytest.approx(counts_per_second, rel=1e-9)
|
||||||
|
assert fit.axis_overhead_s == pytest.approx(overhead, rel=1e-9)
|
||||||
|
# Two points fit both terms exactly, so there is nothing left over -- and
|
||||||
|
# that is precisely why the residual cannot be used as a check.
|
||||||
|
assert fit.residual_ppm == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_fit_timing_least_squares_over_more_points():
|
||||||
|
counts_per_second, overhead = 92889.0, 38.11e-6
|
||||||
|
ccs = [50, 100, 200, 400, 800]
|
||||||
|
periods = [planted_period(cc, counts_per_second, overhead) for cc in ccs]
|
||||||
|
fit = calibrate.fit_timing(ccs, periods)
|
||||||
|
assert fit.counts_per_second == pytest.approx(counts_per_second, rel=1e-6)
|
||||||
|
assert fit.axis_overhead_s == pytest.approx(overhead, rel=1e-6)
|
||||||
|
|
||||||
|
|
||||||
|
def test_fit_timing_needs_two_distinct_cycle_counts():
|
||||||
|
"""Two unknowns need two points; one would be a guess dressed as a fit."""
|
||||||
|
with pytest.raises(calibrate.CalibrationError,
|
||||||
|
match="two distinct cycle counts"):
|
||||||
|
calibrate.fit_timing([100, 100, 100], [3.5e-3] * 3, period_sd=1e-4)
|
||||||
|
|
||||||
|
|
||||||
|
def test_fit_timing_rejects_data_that_does_not_follow_the_model():
|
||||||
|
"""A higher cycle count that samples faster is not this chip."""
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="not positive"):
|
||||||
|
calibrate.fit_timing([100, 400], [13.0e-3, 3.5e-3], period_sd=1e-4)
|
||||||
|
|
||||||
|
|
||||||
|
def test_fit_timing_rejects_mismatched_lengths():
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="against"):
|
||||||
|
calibrate.fit_timing([100, 400], [3.5e-3], period_sd=1e-4)
|
||||||
|
|
||||||
|
|
||||||
|
def test_holding_the_divisor_at_spec_cannot_absorb_the_misfit():
|
||||||
|
"""The reason both terms are calibrated, not just the overhead.
|
||||||
|
|
||||||
|
Solving for the overhead alone with the spec divisor gives a different
|
||||||
|
answer at every cycle count, which is what makes the nominal model's error
|
||||||
|
cycle-count dependent.
|
||||||
|
"""
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
solved = [planted_period(cc, counts_per_second, overhead) / rm3100.AXES
|
||||||
|
- cc / rm3100.COUNTS_PER_SECOND for cc in (100, 400)]
|
||||||
|
assert solved[0] == pytest.approx(58.9e-6, abs=1e-6)
|
||||||
|
assert solved[1] == pytest.approx(113.6e-6, abs=1e-6)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# oscillator_hz
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("cycle_count", [50, 100, 200, 400, 800])
|
||||||
|
def test_oscillator_is_recovered_from_a_single_capture(tmp_path, cycle_count):
|
||||||
|
"""One capture is enough once the overhead is known -- the whole point."""
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=cycle_count,
|
||||||
|
counts_per_second=counts_per_second, overhead=overhead)
|
||||||
|
cal = calibrate.calibration(counts_per_second, overhead)
|
||||||
|
assert calibrate.oscillator_hz(cap, cal).value == pytest.approx(
|
||||||
|
counts_per_second, rel=1e-5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_overhead_error_costs_less_than_a_tenth_of_a_percent(tmp_path):
|
||||||
|
"""The sensitivity the single-capture path depends on.
|
||||||
|
|
||||||
|
A 1 us error in the overhead must stay under 0.1% of the oscillator at the
|
||||||
|
worst cycle count in use, or a calibration from one supply could not be
|
||||||
|
applied to a capture from another.
|
||||||
|
"""
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=100,
|
||||||
|
counts_per_second=counts_per_second, overhead=overhead)
|
||||||
|
exact = calibrate.oscillator_hz(
|
||||||
|
cap, calibrate.calibration(counts_per_second, overhead)).value
|
||||||
|
off_by_1us = calibrate.oscillator_hz(
|
||||||
|
cap, calibrate.calibration(counts_per_second, overhead + 1e-6)).value
|
||||||
|
assert abs(off_by_1us / exact - 1) < 0.001
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_nominal_overhead_costs_far_more_and_varies_with_cycle_count(tmp_path):
|
||||||
|
"""Why the nominal model cannot be used as a gain reference."""
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
errors = []
|
||||||
|
for cycle_count in (100, 400):
|
||||||
|
cap = make(tmp_path / f"{cycle_count}.csv", cycle_count=cycle_count,
|
||||||
|
counts_per_second=counts_per_second, overhead=overhead)
|
||||||
|
nominal = calibrate.calibration(counts_per_second,
|
||||||
|
rm3100.AXIS_OVERHEAD_S)
|
||||||
|
errors.append(calibrate.oscillator_hz(cap, nominal).value
|
||||||
|
/ counts_per_second)
|
||||||
|
# Both wrong, and wrong by different amounts -- so it does not cancel.
|
||||||
|
assert abs(errors[0] - 1) > 0.02
|
||||||
|
assert abs(errors[1] - 1) < 0.01
|
||||||
|
|
||||||
|
|
||||||
|
def test_oscillator_refuses_a_calibration_that_cannot_describe_the_capture(tmp_path):
|
||||||
|
"""An overhead longer than the whole per-axis time is not merely wrong."""
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=100)
|
||||||
|
absurd = calibrate.calibration(88546.0, axis_overhead_s=1.0)
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="does not describe"):
|
||||||
|
calibrate.oscillator_hz(cap, absurd)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# gain_factor
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_a_capture_at_its_own_reference_is_not_corrected(tmp_path):
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
cap = make(tmp_path / "c.csv", counts_per_second=counts_per_second,
|
||||||
|
overhead=overhead)
|
||||||
|
cal = calibrate.calibration(counts_per_second, overhead)
|
||||||
|
assert calibrate.gain_factor(cap, cal).value == pytest.approx(1.0, rel=1e-5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_gain_factor_is_the_oscillator_ratio_at_exponent_one(tmp_path):
|
||||||
|
overhead = 40.61e-6
|
||||||
|
cap = make(tmp_path / "c.csv", counts_per_second=92889.0, overhead=overhead)
|
||||||
|
cal = calibrate.calibration(88546.0, overhead, reference_oscillator_hz=88546.0)
|
||||||
|
assert calibrate.gain_factor(cap, cal).value == pytest.approx(
|
||||||
|
92889.0 / 88546.0, rel=1e-4)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_zero_exponent_disables_the_oscillator_term(tmp_path):
|
||||||
|
overhead = 40.61e-6
|
||||||
|
cap = make(tmp_path / "c.csv", counts_per_second=92889.0, overhead=overhead)
|
||||||
|
cal = calibrate.calibration(88546.0, overhead, gain_exponent=0.0)
|
||||||
|
assert calibrate.gain_factor(cap, cal).value == 1.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_correction_is_exactly_one_at_the_reference_cycle_count(tmp_path):
|
||||||
|
"""The offset is only known up to a scale, so the reference pins it."""
|
||||||
|
counts_per_second, overhead = 88546.0, 40.61e-6
|
||||||
|
for reference in (50, 100, 400):
|
||||||
|
cap = make(tmp_path / f"{reference}.csv", cycle_count=reference,
|
||||||
|
counts_per_second=counts_per_second, overhead=overhead)
|
||||||
|
for offset in (-0.5, 0.0, 0.9, 4.086):
|
||||||
|
cal = calibrate.calibration(
|
||||||
|
counts_per_second, overhead, reference_cycle_count=reference,
|
||||||
|
gain_offset_counts=offset)
|
||||||
|
assert calibrate.gain_factor(cap, cal).value == pytest.approx(
|
||||||
|
1.0, rel=1e-5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_offset_corrects_the_datasheet_gain_shape(tmp_path):
|
||||||
|
"""The datasheet implies gain ~ (cc + 4.086); this unit measures ~0.
|
||||||
|
|
||||||
|
Two captures whose counts embed gain = A(cc + n) must read the same field
|
||||||
|
once corrected with that n, and must not with the datasheet's.
|
||||||
|
"""
|
||||||
|
counts_per_second, overhead, n = 88546.0, 40.61e-6, 0.0
|
||||||
|
field = 47000.0
|
||||||
|
raw, good, bad = {}, {}, {}
|
||||||
|
for cycle_count in (100, 400):
|
||||||
|
# The counts this chip would produce if its gain really went as (cc+n).
|
||||||
|
per_axis = field / math.sqrt(3) * 0.3671 * (cycle_count + n)
|
||||||
|
counts = np.tile([per_axis, per_axis, per_axis], (2000, 1))
|
||||||
|
cap = capture.load(write_capture(
|
||||||
|
tmp_path / f"{cycle_count}.csv", rows=2000,
|
||||||
|
cycle_count=cycle_count, counts=counts,
|
||||||
|
dt=planted_period(cycle_count, counts_per_second, overhead)))
|
||||||
|
raw[cycle_count] = cap.total.mean()
|
||||||
|
for name, offset, into in (("good", n, good), ("bad", 4.086, bad)):
|
||||||
|
cal = calibrate.calibration(counts_per_second, overhead,
|
||||||
|
reference_cycle_count=100,
|
||||||
|
gain_offset_counts=offset)
|
||||||
|
into[cycle_count] = cap.total.mean() * calibrate.gain_factor(cap, cal).value
|
||||||
|
|
||||||
|
# Uncorrected the two disagree, because gain_model has the wrong offset.
|
||||||
|
assert abs(raw[400] / raw[100] - 1) > 0.02
|
||||||
|
assert good[400] == pytest.approx(good[100], rel=1e-4)
|
||||||
|
assert abs(bad[400] / bad[100] - 1) > 0.02
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_gain_offset_that_zeroes_the_gain_is_refused(tmp_path):
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=100)
|
||||||
|
cal = calibrate.calibration(88546.0, 40.61e-6, reference_cycle_count=100,
|
||||||
|
gain_offset_counts=-100.0)
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="at or below zero"):
|
||||||
|
calibrate.gain_factor(cap, cal)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Propagated uncertainty
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_zero_input_uncertainty_gives_zero_output_uncertainty(tmp_path):
|
||||||
|
"""A calibration claiming perfect knowledge says so, rather than guessing."""
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=400)
|
||||||
|
cal = calibrate.calibration(88546.0, 40.61e-6)
|
||||||
|
assert calibrate.gain_factor(cap, cal, period_sd=0.0).sd == 0.0
|
||||||
|
assert calibrate.oscillator_hz(cap, cal, period_sd=0.0).sd == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_offset_uncertainty_vanishes_at_the_reference_cycle_count(tmp_path):
|
||||||
|
"""At the reference the shape term is 1 by construction, so it contributes
|
||||||
|
nothing however badly the offset is known."""
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=100)
|
||||||
|
cal = calibrate.calibration(88546.0, 40.61e-6, reference_cycle_count=100,
|
||||||
|
gain_offset_counts_sd=10.0)
|
||||||
|
assert calibrate.gain_factor(cap, cal, period_sd=0.0).sd == pytest.approx(0.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_offset_uncertainty_scales_linearly_away_from_the_reference(tmp_path):
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=400)
|
||||||
|
one = calibrate.calibration(88546.0, 40.61e-6, reference_cycle_count=100,
|
||||||
|
gain_offset_counts_sd=0.5)
|
||||||
|
two = calibrate.calibration(88546.0, 40.61e-6, reference_cycle_count=100,
|
||||||
|
gain_offset_counts_sd=1.0)
|
||||||
|
a = calibrate.gain_factor(cap, one, period_sd=0.0)
|
||||||
|
b = calibrate.gain_factor(cap, two, period_sd=0.0)
|
||||||
|
assert b.sd == pytest.approx(2 * a.sd, rel=1e-9)
|
||||||
|
assert a.relative > 0.001 # and it is not negligible
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_oscillator_term_scales_with_the_exponent(tmp_path):
|
||||||
|
cap = make(tmp_path / "c.csv", cycle_count=100, counts_per_second=92889.0)
|
||||||
|
one = calibrate.calibration(88546.0, 40.61e-6, axis_overhead_s_sd=1e-6,
|
||||||
|
gain_exponent=1.0)
|
||||||
|
two = calibrate.calibration(88546.0, 40.61e-6, axis_overhead_s_sd=1e-6,
|
||||||
|
gain_exponent=2.0)
|
||||||
|
a = calibrate.gain_factor(cap, one, period_sd=0.0)
|
||||||
|
b = calibrate.gain_factor(cap, two, period_sd=0.0)
|
||||||
|
assert b.relative == pytest.approx(2 * a.relative, rel=1e-9)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_stability_is_zero_for_a_constant_period(tmp_path):
|
||||||
|
"""Not exactly zero: the capture format stores host time to a microsecond,
|
||||||
|
which is ~0.05 ppm of scatter here. Real drifts run 186 to 1241 ppm, so a
|
||||||
|
1 ppm bar separates "nothing" from anything worth reporting."""
|
||||||
|
cap = make(tmp_path / "c.csv", rows=4000)
|
||||||
|
assert calibrate.rate_stability(cap) < 1e-6
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_stability_recovers_a_planted_ramp(tmp_path):
|
||||||
|
"""The oscillator warms up and slows; this is what measures that."""
|
||||||
|
rows, dt, ramp = 8000, 1 / 300.0, 0.002 # 2000 ppm across the run
|
||||||
|
index = np.arange(rows)
|
||||||
|
# Period growing linearly, so the instantaneous rate falls by `ramp`.
|
||||||
|
times = np.cumsum(dt * (1 + ramp * index / rows))
|
||||||
|
path = tmp_path / "c.csv"
|
||||||
|
write_capture(path, rows=rows, dt=dt)
|
||||||
|
text = path.read_text().splitlines()
|
||||||
|
header = [l for l in text if l.startswith("#")] + [text[len(
|
||||||
|
[l for l in text if l.startswith("#")])]]
|
||||||
|
body = text[len(header):]
|
||||||
|
rebuilt = header + [
|
||||||
|
",".join([row.split(",")[0], f"{1_700_000_000.0 + t:.6f}"]
|
||||||
|
+ row.split(",")[2:])
|
||||||
|
for row, t in zip(body, times)]
|
||||||
|
path.write_text("\n".join(rebuilt) + "\n")
|
||||||
|
cap = capture.load(path)
|
||||||
|
assert calibrate.rate_stability(cap) == pytest.approx(ramp, rel=0.15)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_two_point_fit_refuses_to_invent_a_confidence():
|
||||||
|
"""It has no residual, so it cannot estimate its own uncertainty."""
|
||||||
|
ccs = [100, 400]
|
||||||
|
periods = [planted_period(cc, 88546.0, 40.61e-6) for cc in ccs]
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="two-point fit"):
|
||||||
|
calibrate.fit_timing(ccs, periods)
|
||||||
|
|
||||||
|
|
||||||
|
def test_three_points_carry_a_residual_that_can_catch_a_bad_one():
|
||||||
|
clean = [planted_period(cc, 88546.0, 40.61e-6) for cc in (100, 200, 400)]
|
||||||
|
good = calibrate.fit_timing([100, 200, 400], clean)
|
||||||
|
assert good.residual_ppm < 1.0
|
||||||
|
nudged = list(clean)
|
||||||
|
nudged[1] *= 1.002
|
||||||
|
bad = calibrate.fit_timing([100, 200, 400], nudged)
|
||||||
|
assert bad.residual_ppm > 100.0
|
||||||
|
assert bad.counts_per_second_sd > good.counts_per_second_sd
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# The calibration file
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_a_calibration_round_trips(tmp_path):
|
||||||
|
cal = calibrate.calibration(
|
||||||
|
88546.0, 40.61e-6, reference_oscillator_hz=88000.0,
|
||||||
|
reference_cycle_count=200, gain_exponent=1.4, gain_offset_counts=0.9,
|
||||||
|
counts_per_second_sd=70.0, axis_overhead_s_sd=1.1e-6,
|
||||||
|
gain_exponent_sd=0.2, gain_offset_counts_sd=0.5,
|
||||||
|
note="bench", created="2026-08-24")
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
calibrate.save_calibration(path, cal)
|
||||||
|
assert calibrate.load_calibration(path) == cal
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_reference_defaults_to_the_measured_oscillator():
|
||||||
|
"""A calibration corrects nothing at the condition it was taken at."""
|
||||||
|
cal = calibrate.calibration(88546.0, 40.61e-6)
|
||||||
|
assert cal.reference_oscillator_hz == 88546.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_missing_version_is_refused(tmp_path):
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
path.write_text(json.dumps({"counts_per_second": 1.0,
|
||||||
|
"axis_overhead_s": 0.0}))
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="rm3100_calibration"):
|
||||||
|
calibrate.load_calibration(path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_future_version_is_refused_rather_than_guessed(tmp_path):
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="rm3100_calibration"):
|
||||||
|
calibrate.load_calibration(
|
||||||
|
written(tmp_path, rm3100_calibration=calibrate.CALIBRATION_VERSION + 1))
|
||||||
|
|
||||||
|
|
||||||
|
def test_truncated_json_is_refused(tmp_path):
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
path.write_text('{"rm3100_calibration": 1, "counts_per_second":')
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="not valid JSON"):
|
||||||
|
calibrate.load_calibration(path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_json_list_is_refused(tmp_path):
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
path.write_text("[1, 2, 3]")
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="expected an object"):
|
||||||
|
calibrate.load_calibration(path)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("field", ["counts_per_second", "axis_overhead_s"])
|
||||||
|
def test_a_missing_required_field_names_itself(tmp_path, field):
|
||||||
|
body = {"rm3100_calibration": calibrate.CALIBRATION_VERSION,
|
||||||
|
"counts_per_second": 88546.0, "axis_overhead_s": 40.61e-6}
|
||||||
|
del body[field]
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
path.write_text(json.dumps(body))
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match=field):
|
||||||
|
calibrate.load_calibration(path)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("bad", [0, -1, "abc", None, True, [1], float("nan")])
|
||||||
|
def test_a_bad_count_rate_is_refused(tmp_path, bad):
|
||||||
|
path = tmp_path / "cal.json"
|
||||||
|
# json.dump cannot write nan as valid JSON, so write it literally.
|
||||||
|
body = ('{"rm3100_calibration": 2, "axis_overhead_s": 4e-05, '
|
||||||
|
f'"counts_per_second": {json.dumps(bad) if bad == bad else "NaN"}}}')
|
||||||
|
path.write_text(body)
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="counts_per_second"):
|
||||||
|
calibrate.load_calibration(path)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("bad", ["abc", None, True])
|
||||||
|
def test_a_bad_exponent_is_refused(tmp_path, bad):
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="gain_exponent"):
|
||||||
|
calibrate.load_calibration(written(tmp_path, gain_exponent=bad))
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_non_integer_reference_cycle_count_is_refused(tmp_path):
|
||||||
|
with pytest.raises(calibrate.CalibrationError,
|
||||||
|
match="reference_cycle_count"):
|
||||||
|
calibrate.load_calibration(
|
||||||
|
written(tmp_path, reference_cycle_count="fast"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_negative_uncertainty_is_refused(tmp_path):
|
||||||
|
with pytest.raises(calibrate.CalibrationError,
|
||||||
|
match="gain_offset_counts_sd"):
|
||||||
|
calibrate.load_calibration(
|
||||||
|
written(tmp_path, gain_offset_counts_sd=-1.0))
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_version_1_file_is_refused_with_an_explanation(tmp_path):
|
||||||
|
"""Version 1's per-cycle-count table has no version 2 equivalent."""
|
||||||
|
path = tmp_path / "old.json"
|
||||||
|
path.write_text(json.dumps({"rm3100_calibration": 1,
|
||||||
|
"counts_per_second": 88546.0,
|
||||||
|
"axis_overhead_s": 40.61e-6,
|
||||||
|
"gain_scale_by_cycle_count": {"400": 0.977}}))
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="re-measure"):
|
||||||
|
calibrate.load_calibration(path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_true_is_not_accepted_as_a_number(tmp_path):
|
||||||
|
"""JSON's true floats to 1.0 in Python, which would pass silently."""
|
||||||
|
with pytest.raises(calibrate.CalibrationError, match="counts_per_second"):
|
||||||
|
calibrate.load_calibration(written(tmp_path, counts_per_second=True))
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Conversion and provenance
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def read_calibrated(path):
|
||||||
|
"""Parse a calibrated CSV back into (header dict, rows)."""
|
||||||
|
meta, rows = {}, []
|
||||||
|
with open(path) as handle:
|
||||||
|
for line in handle:
|
||||||
|
if line.startswith("#"):
|
||||||
|
key, _, value = line[1:].partition(":")
|
||||||
|
meta[key.strip()] = value.strip()
|
||||||
|
else:
|
||||||
|
rows.append(line.rstrip("\n").split(","))
|
||||||
|
return meta, rows[0], rows[1:]
|
||||||
|
|
||||||
|
|
||||||
|
def test_uncorrected_output_matches_capture_exactly(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
cap = make(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source)
|
||||||
|
meta, header, rows = read_calibrated(out)
|
||||||
|
assert meta["gain_factor"] == "1.0"
|
||||||
|
assert meta["calibration"] == "none"
|
||||||
|
assert header == ["sample_index", "elapsed_s",
|
||||||
|
"x_nT", "y_nT", "z_nT", "total_nT", "warning"]
|
||||||
|
assert len(rows) == len(cap.sample_index)
|
||||||
|
assert float(rows[0][2]) == pytest.approx(cap.x[0], abs=5e-4)
|
||||||
|
assert float(rows[0][5]) == pytest.approx(cap.total[0], abs=5e-4)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_total_column_is_the_norm_of_the_three(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
cap = make(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source, factor=1.049)
|
||||||
|
_, _, rows = read_calibrated(out)
|
||||||
|
for row in rows[:50]:
|
||||||
|
x, y, z, total = (float(v) for v in row[2:6])
|
||||||
|
assert total == pytest.approx(math.sqrt(x*x + y*y + z*z), abs=1e-2)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_factor_scales_the_field(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
cap = make(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source, factor=2.0)
|
||||||
|
_, _, rows = read_calibrated(out)
|
||||||
|
assert float(rows[0][2]) == pytest.approx(cap.x[0] * 2.0, abs=1e-3)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_source_digest_matches_and_changes_with_the_source(tmp_path):
|
||||||
|
"""The answer to a derived file drifting from its source unnoticed."""
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
cap = make(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source)
|
||||||
|
meta, _, _ = read_calibrated(out)
|
||||||
|
assert meta["source_sha256"] == calibrate.source_digest(source)
|
||||||
|
|
||||||
|
source.write_text(source.read_text() + "\n")
|
||||||
|
assert meta["source_sha256"] != calibrate.source_digest(source)
|
||||||
|
|
||||||
|
|
||||||
|
def test_flags_survive_the_conversion(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
write_capture(source, rows=200,
|
||||||
|
flags={40: capture.WARN_MISSED,
|
||||||
|
41: f"{capture.WARN_MISSED} {capture.WARN_AMBIGUOUS}",
|
||||||
|
42: capture.WARN_AMBIGUOUS})
|
||||||
|
cap = capture.load(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source)
|
||||||
|
_, _, rows = read_calibrated(out)
|
||||||
|
assert rows[40][6] == capture.WARN_MISSED
|
||||||
|
assert rows[41][6] == f"{capture.WARN_MISSED} {capture.WARN_AMBIGUOUS}"
|
||||||
|
assert rows[42][6] == capture.WARN_AMBIGUOUS
|
||||||
|
assert rows[39][6] == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_calibrated_file_is_not_a_capture(tmp_path):
|
||||||
|
"""Different format, and capture.py must say so rather than misread it."""
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
cap = make(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source)
|
||||||
|
with pytest.raises(capture.CaptureError, match="no capture header"):
|
||||||
|
capture.load(out)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_header_records_the_calibration_that_was_applied(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
cap = make(source, counts_per_second=92889.0, overhead=40.61e-6)
|
||||||
|
cal = calibrate.calibration(88546.0, 40.61e-6)
|
||||||
|
cal_path = tmp_path / "bench.json"
|
||||||
|
calibrate.save_calibration(cal_path, cal)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
calibrate.write_calibrated(cap, out, source, cal,
|
||||||
|
calibrate.gain_factor(cap, cal), cal_path)
|
||||||
|
meta, _, _ = read_calibrated(out)
|
||||||
|
assert meta["calibration"] == "bench.json"
|
||||||
|
assert float(meta["gain_factor"]) == pytest.approx(92889.0 / 88546.0,
|
||||||
|
rel=1e-4)
|
||||||
|
assert float(meta["oscillator_hz"]) == pytest.approx(92889.0, rel=1e-4)
|
||||||
|
assert float(meta["reference_oscillator_hz"]) == 88546.0
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# End to end
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def run(monkeypatch, capsys, argv):
|
||||||
|
monkeypatch.setattr("sys.argv", ["calibrate.py"] + argv)
|
||||||
|
assert calibrate.main() == 0
|
||||||
|
return capsys.readouterr()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_converts_without_a_calibration(tmp_path, monkeypatch, capsys):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
make(source)
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
result = run(monkeypatch, capsys, [str(source), "-o", str(out)])
|
||||||
|
assert "correcting nothing" in result.out
|
||||||
|
assert out.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_applies_a_calibration(tmp_path, monkeypatch, capsys):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
make(source, counts_per_second=92889.0, overhead=40.61e-6)
|
||||||
|
cal_path = tmp_path / "bench.json"
|
||||||
|
calibrate.save_calibration(cal_path,
|
||||||
|
calibrate.calibration(88546.0, 40.61e-6))
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
result = run(monkeypatch, capsys,
|
||||||
|
[str(source), "-o", str(out), "--calibration", str(cal_path)])
|
||||||
|
assert "gain factor 1.04" in result.out
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_fails_before_reading_when_the_output_is_unwritable(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
make(source)
|
||||||
|
import sys as _sys
|
||||||
|
argv = [str(source), "-o", str(tmp_path / "nope" / "out.csv")]
|
||||||
|
_sys.argv = ["calibrate.py"] + argv
|
||||||
|
with pytest.raises(SystemExit, match="not a directory"):
|
||||||
|
calibrate.main()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_reports_a_bad_calibration_rather_than_writing_output(tmp_path):
|
||||||
|
source = tmp_path / "raw.csv"
|
||||||
|
make(source)
|
||||||
|
bad = tmp_path / "bad.json"
|
||||||
|
bad.write_text("{}")
|
||||||
|
out = tmp_path / "out.csv"
|
||||||
|
import sys as _sys
|
||||||
|
_sys.argv = ["calibrate.py", str(source), "-o", str(out),
|
||||||
|
"--calibration", str(bad)]
|
||||||
|
with pytest.raises(SystemExit, match="rm3100_calibration"):
|
||||||
|
calibrate.main()
|
||||||
|
assert not out.exists()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Measuring the oscillator's drift
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
DRIFT_DT = 1.0 / 250.0
|
||||||
|
# 102 s: ten 10 s windows with room to spare. Landing exactly on a multiple of
|
||||||
|
# the window would make the window count depend on whether the planted drift
|
||||||
|
# happens to push the duration over it.
|
||||||
|
DRIFT_ROWS = 25_500
|
||||||
|
|
||||||
|
|
||||||
|
def drifting(path, ppm_per_second=0.0, jitter_s=0.0, rows=DRIFT_ROWS,
|
||||||
|
dt=DRIFT_DT, seed=0):
|
||||||
|
"""A capture whose sample period ramps, with optional host-side jitter."""
|
||||||
|
return capture.load(write_capture(
|
||||||
|
path, rows=rows, dt=dt, counts=field_counts(rows, FIELD, 100, 5.0, seed),
|
||||||
|
times=drifting_times(rows, dt, ppm_per_second, jitter_s, seed)))
|
||||||
|
|
||||||
|
|
||||||
|
def test_window_rates_recovers_a_planted_ramp(tmp_path):
|
||||||
|
"""The rate falls at the planted rate, in fractional terms per second."""
|
||||||
|
ppm_per_second = 8.0
|
||||||
|
cap = drifting(tmp_path / "d.csv", ppm_per_second=ppm_per_second)
|
||||||
|
centres, rates = calibrate.window_rates(cap, 10.0)
|
||||||
|
assert len(rates) == 10
|
||||||
|
# A period ramping up as (1 + r*t) is a rate falling as (1 - r*t).
|
||||||
|
slope = np.polyfit(centres, rates / rates.mean(), 1)[0]
|
||||||
|
assert slope == pytest.approx(-ppm_per_second * 1e-6, rel=0.05)
|
||||||
|
|
||||||
|
|
||||||
|
def test_window_rates_is_flat_without_drift(tmp_path):
|
||||||
|
cap = drifting(tmp_path / "f.csv")
|
||||||
|
_, rates = calibrate.window_rates(cap, 10.0)
|
||||||
|
assert np.ptp(rates) / rates.mean() < 1e-9
|
||||||
|
|
||||||
|
|
||||||
|
def test_window_rates_centres_span_the_capture(tmp_path):
|
||||||
|
cap = drifting(tmp_path / "c.csv")
|
||||||
|
centres, rates = calibrate.window_rates(cap, 10.0)
|
||||||
|
assert len(centres) == len(rates)
|
||||||
|
assert centres[0] == pytest.approx(5.0)
|
||||||
|
assert centres[-1] == pytest.approx(95.0)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("window", [0.0, -1.0, 60.0, 1e6])
|
||||||
|
def test_window_rates_declines_impossible_windows(tmp_path, window):
|
||||||
|
"""Non-positive, or too long to fit two windows in the capture."""
|
||||||
|
cap = drifting(tmp_path / "s.csv")
|
||||||
|
centres, rates = calibrate.window_rates(cap, window)
|
||||||
|
assert len(centres) == 0 and len(rates) == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_stability_recovers_a_planted_ramp(tmp_path):
|
||||||
|
"""Peak to peak across eight windows, against the ramp that produced it."""
|
||||||
|
cap = drifting(tmp_path / "r.csv", ppm_per_second=8.0)
|
||||||
|
# Eight windows of 12.5 s: centres 6.25 s and 93.75 s apart, so the spread
|
||||||
|
# is the ramp over 87.5 s, not over the full 100 s.
|
||||||
|
expected = 8.0e-6 * cap.duration * (1 - 1 / 8)
|
||||||
|
assert calibrate.rate_stability(cap) == pytest.approx(expected, rel=0.05)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_stability_is_zero_on_a_uniform_grid(tmp_path):
|
||||||
|
assert calibrate.rate_stability(drifting(tmp_path / "u.csv")) < 1e-9
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_stability_declines_a_capture_too_short_to_window(tmp_path):
|
||||||
|
cap = drifting(tmp_path / "t.csv", rows=200)
|
||||||
|
assert calibrate.rate_stability(cap) == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_allan_of_a_linear_drift_rises_with_tau(tmp_path):
|
||||||
|
"""sigma_y(tau) = |D| * tau / sqrt(2) for a deterministic frequency ramp.
|
||||||
|
|
||||||
|
This is the shape the real captures show above a few seconds, and it is what
|
||||||
|
separates a warming oscillator from a random walk.
|
||||||
|
"""
|
||||||
|
ppm_per_second = 8.0
|
||||||
|
cap = drifting(tmp_path / "a.csv", ppm_per_second=ppm_per_second)
|
||||||
|
taus, devs = calibrate.rate_allan(cap, [4.0, 8.0, 16.0])
|
||||||
|
assert list(taus) == [4.0, 8.0, 16.0]
|
||||||
|
for tau, dev in zip(taus, devs):
|
||||||
|
assert dev == pytest.approx(ppm_per_second * 1e-6 * tau / math.sqrt(2),
|
||||||
|
rel=0.05)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_allan_falls_with_tau_when_only_jitter_is_present(tmp_path):
|
||||||
|
"""Independent read jitter averages down; drift does not. Opposite slopes."""
|
||||||
|
cap = drifting(tmp_path / "j.csv", jitter_s=2e-3)
|
||||||
|
taus, devs = calibrate.rate_allan(cap, [4.0, 16.0])
|
||||||
|
assert devs[0] > devs[1] * 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_allan_is_negligible_on_a_uniform_grid(tmp_path):
|
||||||
|
cap = drifting(tmp_path / "z.csv")
|
||||||
|
_, devs = calibrate.rate_allan(cap, [4.0, 8.0])
|
||||||
|
assert np.all(devs < 1e-9)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_allan_drops_taus_that_yield_too_few_windows(tmp_path):
|
||||||
|
"""Two windows give one difference, which is not an estimate of anything."""
|
||||||
|
cap = drifting(tmp_path / "d2.csv")
|
||||||
|
taus, devs = calibrate.rate_allan(cap, [10.0, 40.0, 200.0])
|
||||||
|
assert list(taus) == [10.0] # 40 s gives two windows, 200 s none
|
||||||
|
assert len(devs) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_allan_accepts_an_empty_request(tmp_path):
|
||||||
|
taus, devs = calibrate.rate_allan(drifting(tmp_path / "e.csv"), [])
|
||||||
|
assert len(taus) == 0 and len(devs) == 0
|
||||||
|
|
@ -324,9 +324,13 @@ def test_fit_rate_model_is_exact_from_two_cycle_counts(tmp_path):
|
||||||
caps = [make(tmp_path / f"{cc}.csv", rows=3000, cycle_count=cc,
|
caps = [make(tmp_path / f"{cc}.csv", rows=3000, cycle_count=cc,
|
||||||
dt=planted_period(cc, count_rate, overhead))
|
dt=planted_period(cc, count_rate, overhead))
|
||||||
for cc in (100, 400)]
|
for cc in (100, 400)]
|
||||||
fitted_rate, fitted_overhead = compare.fit_rate_model(caps)
|
fit = compare.fit_rate_model(caps)
|
||||||
assert fitted_rate == pytest.approx(count_rate, rel=1e-4)
|
assert fit.counts_per_second == pytest.approx(count_rate, rel=1e-4)
|
||||||
assert fitted_overhead == pytest.approx(overhead, rel=1e-3)
|
assert fit.axis_overhead_s == pytest.approx(overhead, rel=1e-3)
|
||||||
|
# Two points fit both terms exactly, so the residual is empty by
|
||||||
|
# construction and cannot be used to detect a bad point.
|
||||||
|
assert fit.residual_ppm == 0.0
|
||||||
|
assert fit.points == 2
|
||||||
|
|
||||||
|
|
||||||
def test_fit_rate_model_least_squares_over_three_cycle_counts(tmp_path):
|
def test_fit_rate_model_least_squares_over_three_cycle_counts(tmp_path):
|
||||||
|
|
@ -334,9 +338,10 @@ def test_fit_rate_model_least_squares_over_three_cycle_counts(tmp_path):
|
||||||
caps = [make(tmp_path / f"{cc}.csv", rows=3000, cycle_count=cc,
|
caps = [make(tmp_path / f"{cc}.csv", rows=3000, cycle_count=cc,
|
||||||
dt=planted_period(cc, count_rate, overhead))
|
dt=planted_period(cc, count_rate, overhead))
|
||||||
for cc in (100, 200, 400)]
|
for cc in (100, 200, 400)]
|
||||||
fitted_rate, fitted_overhead = compare.fit_rate_model(caps)
|
fit = compare.fit_rate_model(caps)
|
||||||
assert fitted_rate == pytest.approx(count_rate, rel=1e-3)
|
assert fit.counts_per_second == pytest.approx(count_rate, rel=1e-3)
|
||||||
assert fitted_overhead == pytest.approx(overhead, rel=1e-2)
|
assert fit.axis_overhead_s == pytest.approx(overhead, rel=1e-2)
|
||||||
|
assert fit.points == 3
|
||||||
|
|
||||||
|
|
||||||
def test_fit_rate_model_refuses_one_cycle_count(tmp_path):
|
def test_fit_rate_model_refuses_one_cycle_count(tmp_path):
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue