jeremy/rm3100
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Several corrections and improvements to the code based on report feedback. Cleaned up and consolidated in-code documentation.

Browse source
This commit is contained in:
Jeremy Karst 2026-09-02 13:20:10 -04:00
parent f142b8e6ff
commit 0c053e06f1
16 changed files with 1502 additions and 1957 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
.gitignore vendored
View file

@ -4,4 +4,6 @@ __pycache__
*.png *.png
*.csv *.csv
/Noise_Floor_Testing # Captures, figures and reports stay out of the repo; the lab notebook does not.
/Noise_Floor_Testing/*
!/Noise_Floor_Testing/NOTES.md

307
NOTES.md → Noise_Floor_Testing/NOTES.md
View file

@ -1,8 +1,8 @@
# Measurement notes # Measurement notes
The lab notebook for this rig: what was measured, when, on what, and what it The lab notebook for this rig: what was measured, when, on what, and what it
changed. [README.md](README.md) documents how the code works *now*; this file is changed. [README.md](../README.md) documents how the code works *now*; this file is
the record of how that was arrived at, and it is append-mostly — an entry is the record of how that was arrived at, and it is append-mostly, an entry is
corrected rather than deleted when a later measurement supersedes it. corrected rather than deleted when a later measurement supersedes it.
Every capture cited here predates the current header format and has been Every capture cited here predates the current header format and has been
@ -18,7 +18,7 @@ labelled as such.
## Superseded conclusions ## Superseded conclusions
Read this first — these appear as firm recommendations in earlier entries and Read this first: these appear as firm recommendations in earlier entries and
are no longer right. are no longer right.
| Earlier conclusion | Superseded by | Now | | Earlier conclusion | Superseded by | Now |
@ -26,15 +26,15 @@ are no longer right.
| "400 kHz measured best; use 400" | three-run repetition at each speed | 750 kHz is fastest and is the default. The 4% edge for 400 kHz sat inside the run-to-run spread of a single short run | | "400 kHz measured best; use 400" | three-run repetition at each speed | 750 kHz is fastest and is the default. The 4% edge for 400 kHz sat inside the run-to-run spread of a single short run |
| "Pair cycle count with a TMRC step: 200/0x94, 400/0x95, 800/0x96" | the cycle-count rate model | Hold TMRC fast and use cycle count as the only rate knob. The pairing advice exists only because the rate was being set by TMRC | | "Pair cycle count with a TMRC step: 200/0x94, 400/0x95, 800/0x96" | the cycle-count rate model | Hold TMRC fast and use cycle count as the only rate knob. The pairing advice exists only because the rate was being set by TMRC |
| "Abort the run on a missed measurement" | placeholder rows | The run continues and writes a `MISSED` row, keeping `sample_index` contiguous | | "Abort the run on a missed measurement" | placeholder rows | The run continues and writes a `MISSED` row, keeping `sample_index` contiguous |
| `~84,333 / cc / 3` for the sensor ceiling | fitting overhead separately | `1 / (3 × (cc/90,000 + 68.7 µs))`. The naive constant is not constant — it drifts from 84,429 at cc=100 to 89,191 at cc=1200 | | `~84,333 / cc / 3` for the sensor ceiling | fitting overhead separately | `1 / (3 × (cc/90,000 + 68.7 µs))`. The naive constant is not constant: it drifts from 84,429 at cc=100 to 89,191 at cc=1200 |
| "3.3 V vs 3.0 V — indistinguishable" | interleaved A/B at two cycle counts, 2026-08-23 | Indistinguishable in *noise*, yes. But the **scale differs by 7.1%** and the **sample rate by 4.94%**. The earlier entry could not see it because the sensor had moved and the two effects were confounded | | "3.3 V vs 3.0 V, indistinguishable" | interleaved A/B at two cycle counts, 2026-08-23 | Indistinguishable in *noise*, yes. But the **scale differs by 7.1%** and the **sample rate by 4.94%**. The earlier entry could not see it because the sensor had moved and the two effects were confounded |
| "60 Hz is 25× the noise floor and dominates every sd figure" | the same session, at 285–299 Hz | True at ~35 Hz where it folds to 9 Hz. At the default rate mains is resolved in band at only **1.7–1.8×** the floor. At cc=400 the dominant line is not mains at all but an **fs/4 artefact** | | "60 Hz is 25× the noise floor and dominates every sd figure" | the same session, at 285–299 Hz | True at ~35 Hz where it folds to 9 Hz. At the default rate mains is resolved in band at only **1.7–1.8×** the floor. At cc=400 the dominant line is not mains at all but an **fs/4 artifact** |
| "68.7 µs per-axis overhead" | solving the model per supply | Right in order, wrong in value for this unit: **40.6 µs at 3.006 V, 38.1 µs at 3.353 V**. And the divisor does not hold either — solving both freely gives **88,546 counts/s** at 3.006 V and **92,889** at 3.353 V. Holding it at 90,000 and solving for the overhead alone gives 58.9 µs at cc=100 and 113.6 µs at cc=400, fitting neither. Both terms are per-unit and supply-dependent; `calibrate.py` carries them | | "68.7 µs per-axis overhead" | solving the model per supply | Right in order, wrong in value for this unit: **40.6 µs at 3.006 V, 38.1 µs at 3.353 V**. And the divisor does not hold either: solving both freely gives **88,546 counts/s** at 3.006 V and **92,889** at 3.353 V. Holding it at 90,000 and solving for the overhead alone gives 58.9 µs at cc=100 and 113.6 µs at cc=400, fitting neither. Both terms are per-unit and supply-dependent; `calibrate.py` carries them |
| "`× (1 + rate_error)` corrects gain, cutting the spread to 3.1%" | referencing the correction to a measured oscillator | **The 3.1% was two errors cancelling.** `rate_error` is measured against the *nominal* model, whose error is cycle-count dependent (+0.84% at cc=100, −0.99% at cc=400, same supply), and that partly cancelled the cycle-count gain difference. Referenced properly the figure is **4.9%**. The matched-cycle-count contrasts are unaffected either way | | "`× (1 + rate_error)` corrects gain, cutting the spread to 3.1%" | referencing the correction to a measured oscillator | **The 3.1% was two errors cancelling.** `rate_error` is measured against the *nominal* model, whose error is cycle-count dependent (+0.84% at cc=100, −0.99% at cc=400, same supply), and that partly cancelled the cycle-count gain difference. Referenced properly the figure is **4.9%**. The matched-cycle-count contrasts are unaffected either way |
| "gain ∝ V^−0.65, i.e. rate^−1.47; mechanism unresolved" | freeing the gain offset | **The 1.47 was absorbing a gain-offset error.** Table 3-1's `0.3671·cc + 1.5` implies gain ∝ (cc + 4.086) counts; measured on this unit it is 0.90 (LDO) and −0.09 (3V3). With the offset free, an exponent of exactly **1** is fully consistent — which is what an oscillator-clocked collection interval predicts. Correcting both leaves **2.48%**, inside the 2.3–3.1% movement floor | | "gain ∝ V^−0.65, i.e. rate^−1.47; mechanism unresolved" | freeing the gain offset | **The 1.47 was absorbing a gain-offset error.** Table 3-1's `0.3671·cc + 1.5` implies gain ∝ (cc + 4.086) counts; measured on this unit it is 0.90 (LDO) and −0.09 (3V3). With the offset free, an exponent of exactly **1** is fully consistent, which is what an oscillator-clocked collection interval predicts. Correcting both leaves **2.48%**, inside the 2.3–3.1% movement floor |
| "the per-axis overhead is constant in counts (0.7σ) and changes as a time (2.8σ)" | propagating the thermal drift properly | Those σ came from a half-drift error bar chosen without justification. Propagated, the overhead is 3.60 ± 0.30 counts against 3.54 ± 0.15, and neither reading is excluded. The clean test is the supply rate ratio across cycle counts: oscillator-clocked sits **0.3σ** from it, fixed-time **1.2σ**. Favoured, not proven — drift is the limit | | "the per-axis overhead is constant in counts (0.7σ) and changes as a time (2.8σ)" | propagating the thermal drift properly | Those σ came from a half-drift error bar chosen without justification. Propagated, the overhead is 3.60 ± 0.30 counts against 3.54 ± 0.15, and neither reading is excluded. The clean test is the supply rate ratio across cycle counts: oscillator-clocked sits **0.3σ** from it, fixed-time **1.2σ**. Favoured, not proven: drift is the limit |
| "the within-run drift could be used to correct gain in post-processing" | the coupling test, then σ_y(τ) | **Closed — do not reopen.** Gain does not follow the within-run rate: the measured slopes are **+0.08 to +0.35** where gain ∝ 1/f demands **−1**. The mechanism is that gain is set by the *ratio* of the MagI2C internal clock to f_LR, both on the same die, so a thermal shift is common-mode and divides out. That is what PNI's "stable over temperature" claim in §2 actually protects, and it is why 1241 ppm of rate drift produces no measurable gain drift. The drift is real and well resolved; it simply does not reach the gain | | "the within-run drift could be used to correct gain in post-processing" | the coupling test, then σ_y(τ) | **Closed. Do not reopen.** Gain does not follow the within-run rate: the measured slopes are **+0.08 to +0.35** where gain ∝ 1/f demands **−1**. The mechanism is that gain is set by the *ratio* of the MagI2C internal clock to f_LR, both on the same die, so a thermal shift is common-mode and divides out. That is what PNI's "stable over temperature" claim in §2 actually protects, and it is why 1241 ppm of rate drift produces no measurable gain drift. The drift is real and well resolved; it simply does not reach the gain |
| "the rate measurement is limited by host read jitter, so a longer window is always better" | σ_y(τ) on the four captures | Wrong in both halves. White jitter predicts σ/rate = √12·σ_t/(√n·T), or sub-ppm at 30 s; measured, σ_y bottoms out at **8–14 ppm around τ = 2–10 s** and then *rises*. The floor is correlated host-side noise, 10–30× the naive figure, and past ~10 s a longer window makes the estimate worse, not better — it starts measuring the drift instead of the rate | | "the rate measurement is limited by host read jitter, so a longer window is always better" | σ_y(τ) on the four captures | Wrong in both halves. White jitter predicts σ/rate = √12·σ_t/(√n·T), or sub-ppm at 30 s; measured, σ_y bottoms out at **8–14 ppm around τ = 2–10 s** and then *rises*. The floor is correlated host-side noise, 10–30× the naive figure, and past ~10 s a longer window makes the estimate worse, not better: it starts measuring the drift instead of the rate |
--- ---
@ -56,7 +56,7 @@ are no longer right.
- **Magnitude.** 54.7 µT sits inside the 25–65 µT band for Earth's field, which - **Magnitude.** 54.7 µT sits inside the 25–65 µT band for Earth's field, which
is the real confirmation that the gain and the int24 decode are right. is the real confirmation that the gain and the int24 decode are right.
- **Noise** is well above the 15 nT Table 3-1 quotes for cycle count 200, and is - **Noise** is well above the 15 nT Table 3-1 quotes for cycle count 200, and is
not a driver problem — most likely proximity to a PC and the USB adapter. not a driver problem, most likely proximity to a PC and the USB adapter.
- **Drift.** All three axes fell monotonically over the 10 s (Z most, ~1.3 µT), - **Drift.** All three axes fell monotonically over the 10 s (Z most, ~1.3 µT),
so |B| went from 55.5 to 54.3 µT. Consistent with thermal settling after AVDD so |B| went from 55.5 to 54.3 µT. Consistent with thermal settling after AVDD
is first powered; coil DC resistance moves 0.4 %/°C (Table 3-3). Allow a is first powered; coil DC resistance moves 0.4 %/°C (Table 3-3). Allow a
@ -68,7 +68,7 @@ as Earth's field.
--- ---
## Supply A/B: 3.3 V adapter vs 3.0 V LDO — indistinguishable ## Supply A/B: 3.3 V adapter vs 3.0 V LDO, indistinguishable
> **Superseded in the noise conclusion only, and wrong on scale.** The > **Superseded in the noise conclusion only, and wrong on scale.** The
> interleaved re-run below finds the scale differs by 7.1% and the sample rate by > interleaved re-run below finds the scale differs by 7.1% and the sample rate by
@ -84,8 +84,8 @@ Two 60 s captures on the bench, cycle count 200, TMRC 0x96, 400 kHz.
**Compare noise fractionally, not absolutely.** The measured scale moved 5.9% **Compare noise fractionally, not absolutely.** The measured scale moved 5.9%
between the runs, and a scale change carries the noise with it, so most of the between the runs, and a scale change carries the noise with it, so most of the
absolute sd difference is an artefact. Normalised, the two supplies differ by absolute sd difference is an artifact. Normalised, the two supplies differ by
2.5% — indistinguishable on single 60 s captures. 2.5%, indistinguishable on single 60 s captures.
That is the expected outcome. Table 3-5 allows 50 mV<sub>PP</sub> of supply That is the expected outcome. Table 3-5 allows 50 mV<sub>PP</sub> of supply
ripple and the LP5907's ~6.5 µV RMS sits roughly 1000× inside it, as did the ripple and the LP5907's ~6.5 µV RMS sits roughly 1000× inside it, as did the
@ -100,7 +100,7 @@ translation through a field gradient, so movement and gain are confounded here.
There is still reason to expect 3.0 V to read *more accurately*, but it comes There is still reason to expect 3.0 V to read *more accurately*, but it comes
from the datasheet rather than these captures: Table 3-1 footnote 1 establishes from the datasheet rather than these captures: Table 3-1 footnote 1 establishes
every specification — including the gain fit behind `tesla_per_count` — at every specification, including the gain fit behind `tesla_per_count`, at
3.0 V, noting that performance "will differ" elsewhere. At 3.3 V the gain is an 3.0 V, noting that performance "will differ" elsewhere. At 3.3 V the gain is an
extrapolation off the calibration point. extrapolation off the calibration point.
@ -132,12 +132,12 @@ magnitude difference to gain unless both movement checks pass.
## 60 Hz mains folds into the band ## 60 Hz mains folds into the band
At ~35 Hz the largest spectral feature by far was a line near 9 Hz reaching At ~35 Hz the largest spectral feature by far was a line near 9 Hz reaching
100–125 nT/√Hz against a 4–5 nT/√Hz broadband floor — **25× the noise floor**. 100–125 nT/√Hz against a 4–5 nT/√Hz broadband floor, **25× the noise floor**.
> **The 25× is specific to that rate and that site.** Measured at 285–299 Hz on > **The 25× is specific to that rate and that site.** Measured at 285–299 Hz on
> 2026-08-23, with mains resolved in band rather than folded, the 60 Hz line > 2026-08-23, with mains resolved in band rather than folded, the 60 Hz line
> reaches only **1.7–1.8×** the floor, and at cycle count 400 the dominant line > reaches only **1.7–1.8×** the floor, and at cycle count 400 the dominant line
> is an fs/4 artefact rather than mains. The mechanism below is unchanged; the > is an fs/4 artifact rather than mains. The mechanism below is unchanged; the
> "dominant term" framing is not general. > "dominant term" framing is not general.
It is 60 Hz aliasing, proven by making it move: It is 60 Hz aliasing, proven by making it move:
@ -149,11 +149,11 @@ It is 60 Hz aliasing, proven by making it move:
| long run | 34.5783 Hz | 9.1764 Hz | 9.1566 Hz | | long run | 34.5783 Hz | 9.1764 Hz | 9.1566 Hz |
A real 9 Hz magnetic signal would sit still. This one moves with the sample A real 9 Hz magnetic signal would sit still. This one moves with the sample
rate: `fs` changed by 0.20 Hz and the peak moved 0.43 Hz — the ×2 sensitivity of rate: `fs` changed by 0.20 Hz and the peak moved 0.43 Hz, the ×2 sensitivity of
a second-order fold, exactly as predicted. a second-order fold, exactly as predicted.
The RM3100 has **no anti-alias filter** — cycle-count integration is a boxcar The RM3100 has **no anti-alias filter**: cycle-count integration is a boxcar
with a poor stopband — so out-of-band interference folds in freely. with a poor stopband, so out-of-band interference folds in freely.
**Sampling faster resolves it instead.** At 150 Hz (Nyquist 69.3 Hz) mains lands **Sampling faster resolves it instead.** At 150 Hz (Nyquist 69.3 Hz) mains lands
in band and the strongest line sits at **59.923 Hz**, observed directly. At in band and the strongest line sits at **59.923 Hz**, observed directly. At
@ -167,22 +167,22 @@ filtering can do to an alias.
## Overnight run: 13.2 h, 1.63 M samples ## Overnight run: 13.2 h, 1.63 M samples
Cycle count 200, TMRC 0x96. **Acquisition integrity was perfect** — sample index Cycle count 200, TMRC 0x96. **Acquisition integrity was perfect**: sample index
contiguous across all 1,635,055 rows, system clock monotonic with no NTP steps, contiguous across all 1,635,055 rows, system clock monotonic with no NTP steps,
no malformed rows, no int24 saturation, no missed measurements (worst DRDY no malformed rows, no int24 saturation, no missed measurements (worst DRDY
bracket 11.0 ms of 27.03 ms, 41% of margin). Both problems in the data were bracket 11.0 ms of 27.03 ms, 41% of margin). Both problems in the data were
outside the logger. outside the logger.
**1. The first ~1.5 h and the last ~1 h are contaminated by handling.** |B| **1. The first ~1.5 h and the last ~1 h are contaminated by handling.** |B|
reaches 198,268 nT — five times Earth's field — in bursts around t+0.44 h. reaches 198,268 nT, five times Earth's field, in bursts around t+0.44 h.
Hourly |B| sd tells the story: Hourly |B| sd tells the story:
| hours | \|B\| sd | usable | | hours | \|B\| sd | usable |
|---|---|---| |---|---|---|
| 0–1 | 3,750 nT | no — handling | | 0–1 | 3,750 nT | no, handling |
| 2–10 | **34–40 nT** | yes | | 2–10 | **34–40 nT** | yes |
| 11–12 | 41–68 nT | marginal | | 11–12 | 41–68 nT | marginal |
| 13 | 247 nT | no — activity resumed | | 13 | 247 nT | no, activity resumed |
A 5-minute slice from the middle of the quiet window gave |B| = 42,132 nT with A 5-minute slice from the middle of the quiet window gave |B| = 42,132 nT with
sd 32.5 nT (771 ppm). sd 32.5 nT (771 ppm).
@ -197,19 +197,19 @@ show the oscillator speeding up monotonically:
| 6 | 28.9794 ms | −102 ppm | | 6 | 28.9794 ms | −102 ppm |
| 12 | 28.9187 ms | −2195 ppm | | 12 | 28.9187 ms | −2195 ppm |
That is ~2,500 ppm of rate variation across the quiet hours — thermal, as That is ~2,500 ppm of rate variation across the quiet hours, thermal, as
expected of an RC oscillator. So **long captures cannot be given a single expected of an RC oscillator. So **long captures cannot be given a single
frequency axis**; spectra must be computed on windows short enough that the rate frequency axis**; spectra must be computed on windows short enough that the rate
is constant, via `Capture.restrict()`. is constant, via `Capture.restrict()`.
This is what `drift_limited` and the separate reporting of read jitter and fit This is what `drift_limited` and the separate reporting of read jitter and fit
residual exist for. Over 5 minutes the residual is 3.7 ms; over 8.5 hours it is residual exist for. Over 5 minutes the residual is 3.7 ms; over 8.5 hours it is
2,356 ms — while read jitter stays 0.21 ms in both. Conflating the two would 2,356 ms, while read jitter stays 0.21 ms in both. Conflating the two would
have read as a 20,000× worse host. have read as a 20,000× worse host.
**Drift-limited beyond a second.** A 769 s window of the same run: per-axis sd **Drift-limited beyond a second.** A 769 s window of the same run: per-axis sd
rises to 214–284 nT while the broadband floor barely moves (5.1–5.7 nT/√Hz). The rises to 214–284 nT while the broadband floor barely moves (5.1–5.7 nT/√Hz). The
extra spread is low-frequency drift, not broadband noise — the Allan deviation extra spread is low-frequency drift, not broadband noise: the Allan deviation
bottoms at ~13 nT near τ = 0.12 s and climbs steadily after. bottoms at ~13 nT near τ = 0.12 s and climbs steadily after.
--- ---
@ -220,8 +220,8 @@ Two fixes roughly doubled the achievable rate.
### 1. Two USB round trips per register read → one ### 1. Two USB round trips per register read → one
Reading a register as a pointer write plus a separate read — the form the manual Reading a register as a pointer write plus a separate read, the form the manual
draws (§4.5.2, §5.8.4) — costs two round trips. The part accepts a **repeated draws (§4.5.2, §5.8.4), costs two round trips. The part accepts a **repeated
START**, so `ch347.write_read()` does it in one. Measured at 400 kHz: START**, so `ch347.write_read()` does it in one. Measured at 400 kHz:
| | separate | combined | | | separate | combined |
@ -244,7 +244,7 @@ the default GIL switch interval *is* the jitter floor.
### 3. Splitting the writer off the sampling thread ### 3. Splitting the writer off the sampling thread
The same 60 s capture yielded 22,558 samples (376 Hz) with inline writing and The same 60 s capture yielded 22,558 samples (376 Hz) with inline writing and
28,864 (481 Hz) with the writer split off — about **28%**. 28,864 (481 Hz) with the writer split off, about **28%**.
### Measured ceilings (VM, cycle count 50, 400 kHz) ### Measured ceilings (VM, cycle count 50, 400 kHz)
@ -256,7 +256,7 @@ The same 60 s capture yielded 22,558 samples (376 Hz) with inline writing and
| 0x95 | 75 Hz | clean, 69.2 Hz | 2.63–5.16 ms of 13.33 ms (20–39%) | | 0x95 | 75 Hz | clean, 69.2 Hz | 2.63–5.16 ms of 13.33 ms (20–39%) |
| 0x96 | 37 Hz | clean, 34.7 Hz | 3.30–8.56 ms of 27.03 ms (12–32%) | | 0x96 | 37 Hz | clean, 34.7 Hz | 3.30–8.56 ms of 27.03 ms (12–32%) |
> **These ceilings are host-specific and pessimistic** — VM, virtualised USB, > **These ceilings are host-specific and pessimistic**: VM, virtualised USB,
> shared cores, load average 2.76 at the time of the one 150 Hz failure. The > shared cores, load average 2.76 at the time of the one 150 Hz failure. The
> *shape* carries over (the limit is host round-trip latency, not I²C bandwidth > *shape* carries over (the limit is host round-trip latency, not I²C bandwidth
> or the sensor), but the numbers must be re-measured on the machine that will > or the sensor), but the numbers must be re-measured on the machine that will
@ -265,7 +265,7 @@ The same 60 s capture yielded 22,558 samples (376 Hz) with inline writing and
### The bus is never the constraint ### The bus is never the constraint
Per sample the bus carries a DRDY poll plus a results read — 150 bit-times — Per sample the bus carries a DRDY poll plus a results read, 150 bit-times,
against a host cost 2–3× larger. against a host cost 2–3× larger.
| | 400 kHz | 100 kHz | | | 400 kHz | 100 kHz |
@ -273,10 +273,10 @@ against a host cost 2–3× larger.
| bus time per sample | 0.375 ms | 1.50 ms | | bus time per sample | 0.375 ms | 1.50 ms |
| as % of a 28.86 ms period (37 Hz) | **1.3%** | 5.2% | | as % of a 28.86 ms period (37 Hz) | **1.3%** | 5.2% |
| as % of a 7.21 ms period (150 Hz) | **5.2%** | 20.8% | | as % of a 7.21 ms period (150 Hz) | **5.2%** | 20.8% |
| measured host cost | 0.89–0.98 ms | — | | measured host cost | 0.89–0.98 ms | n/a |
That puts the bus-limited ceiling near 1100 Hz, roughly double the sensor's own That puts the bus-limited ceiling near 1100 Hz, roughly double the sensor's own
562 Hz three-axis maximum at cycle count 50 — so absent host stalls, zero misses 562 Hz three-axis maximum at cycle count 50, so absent host stalls, zero misses
is always achievable. Every miss observed has been a host stall. is always achievable. Every miss observed has been a host stall.
### Bus speed, three runs each at cycle count 228 ### Bus speed, three runs each at cycle count 228
@ -298,7 +298,7 @@ difference was inside the run-to-run spread and does not survive repetition.
## Timing: how the miss detector was arrived at ## Timing: how the miss detector was arrived at
**The read-to-read interval is not a usable threshold.** On the VM it reaches **The read-to-read interval is not a usable threshold.** On the VM it reaches
37 ms against a 28.8 ms period — a 29% overshoot from host stalls alone — while 37 ms against a 28.8 ms period, a 29% overshoot from host stalls alone, while
the DRDY bracket stays under 10 ms. The first implementation thresholded the the DRDY bracket stays under 10 ms. The first implementation thresholded the
interval and aborted a perfectly healthy capture at sample 88. interval and aborted a perfectly healthy capture at sample 88.
@ -310,7 +310,7 @@ out on this unit:
|---|---|---| |---|---|---|
| nominal (TMRC table) | 27.03 ms | −6.4% | | nominal (TMRC table) | 27.03 ms | −6.4% |
| **calibrated (1 s)** | **28.8849 ms** | **0.068%** | | **calibrated (1 s)** | **28.8849 ms** | **0.068%** |
| whole-run fit (25 s) | 28.8653 ms | — | | whole-run fit (25 s) | 28.8653 ms | n/a |
**Counting needs the interval, detection needs the bracket.** Fit residual after **Counting needs the interval, detection needs the bracket.** Fit residual after
adopting the two-signal scheme: adopting the two-signal scheme:
@ -324,7 +324,7 @@ adopting the two-signal scheme:
The load case is the one that shows the design paying off: the bracket reached The load case is the one that shows the design paying off: the bracket reached
119% of threshold, so a stall genuinely occurred, but the interval showed only 119% of threshold, so a stall genuinely occurred, but the interval showed only
one period had elapsed — nothing was lost. The older `int(bracket/period)` one period had elapsed, nothing was lost. The older `int(bracket/period)`
estimator would have inserted a spurious placeholder and slipped the index from estimator would have inserted a spurious placeholder and slipped the index from
there on. At 300 Hz the ambiguous counter fired 19 times. there on. At 300 Hz the ambiguous counter fired 19 times.
@ -349,7 +349,7 @@ is sound over at least a minute.
``` ```
256 Hz is where the VM gives out; everything below it is exact. Noise past cycle 256 Hz is where the VM gives out; everything below it is exact. Noise past cycle
count ~400 is extrapolation — Table 3-1 ends there. count ~400 is extrapolation, Table 3-1 ends there.
**Cycle count sets the ceiling, not TMRC.** Measured at 400 kHz: **Cycle count sets the ceiling, not TMRC.** Measured at 400 kHz:
@ -364,7 +364,7 @@ count ~400 is extrapolation — Table 3-1 ends there.
Table 3-1 gives 440 Hz *single-axis* and says to divide by 3, i.e. ~147 Hz. Table 3-1 gives 440 Hz *single-axis* and says to divide by 3, i.e. ~147 Hz.
Measurement backs Table 3-1 at all three cycle counts. Measurement backs Table 3-1 at all three cycle counts.
**When the cycle count governs, TMRC is irrelevant** — cc=400 gave 73.85 Hz at **When the cycle count governs, TMRC is irrelevant**: cc=400 gave 73.85 Hz at
TMRC 0x92 and 73.86 Hz at 0x94. Varying only cc at TMRC 0x94: TMRC 0x92 and 73.86 Hz at 0x94. Varying only cc at TMRC 0x94:
| cc | measured | governed by | duty | | cc | measured | governed by | duty |
@ -381,7 +381,7 @@ TMRC 0x92 and 73.86 Hz at 0x94. Varying only cc at TMRC 0x94:
| TMRC-governed (cc=50, 0x94) | 138.5 Hz | 50 | **23%** | 3.42 nT/√Hz | | TMRC-governed (cc=50, 0x94) | 138.5 Hz | 50 | **23%** | 3.42 nT/√Hz |
| cycle-count-governed (cc=228) | 128.5 Hz | 228 | **98%** | 2.39 nT/√Hz | | cycle-count-governed (cc=228) | 128.5 Hz | 228 | **98%** | 2.39 nT/√Hz |
Same chip, same field, same per-sample physics — the difference is idle time, Same chip, same field, same per-sample physics, the difference is idle time,
and it is why the current model holds TMRC fast and uses cycle count alone. and it is why the current model holds TMRC fast and uses cycle count alone.
**Equal duty means equal noise.** Two configurations at ~100% duty and very **Equal duty means equal noise.** Two configurations at ~100% duty and very
@ -392,11 +392,11 @@ cc200 @150Hz 138.77 Hz LSB 13.35 nT ASD 2.98 nT/rtHz (66% bracket margin)
cc800 @37Hz 34.72 Hz LSB 3.39 nT ASD 3.26 nT/rtHz (10% bracket margin) cc800 @37Hz 34.72 Hz LSB 3.39 nT ASD 3.26 nT/rtHz (10% bracket margin)
``` ```
Near-identical spectral density, as the equal-duty argument predicts — so the Near-identical spectral density, as the equal-duty argument predicts: so the
choice between them is bandwidth versus resolution, not noise. choice between them is bandwidth versus resolution, not noise.
**Low cycle counts are dominated, not fast.** cc=50 reached the same 138.8 Hz as **Low cycle counts are dominated, not fast.** cc=50 reached the same 138.8 Hz as
cc=200 on the VM — both host-limited, not sensor-limited — while idling 75% of cc=200 on the VM: both host-limited, not sensor-limited, while idling 75% of
the time and quantising 3.8× more coarsely. Its extra sensor speed is capability the time and quantising 3.8× more coarsely. Its extra sensor speed is capability
the host cannot collect. the host cannot collect.
@ -415,13 +415,13 @@ at t ≈ 41 s, so statistics are split around that:
| \|B\| | 369 nT | 358 nT | +313 nT | | \|B\| | 369 nT | 358 nT | +313 nT |
- **The step was the sensor being physically moved**, confirmed at the time, not - **The step was the sensor being physically moved**, confirmed at the time, not
an electrical artefact. A norm shift does not by itself imply an external an electrical artifact. A norm shift does not by itself imply an external
source changed: |B| is preserved under rotation in a uniform field, but a source changed: |B| is preserved under rotation in a uniform field, but a
translation through a gradient samples a different local field. Near a PC the translation through a gradient samples a different local field. Near a PC the
gradients are steep (§4.2.2: field falls off as 1/distance³), so a few gradients are steep (§4.2.2: field falls off as 1/distance³), so a few
centimetres is ample for the +313 nT seen here. Treat a |B| change as "not a centimetres is ample for the +313 nT seen here. Treat a |B| change as "not a
pure rotation", nothing more. pure rotation", nothing more.
- **Noise is strongly axis-dependent** — Y is 4.6× X (388 vs 84 nT) and stable - **Noise is strongly axis-dependent**, Y is 4.6× X (388 vs 84 nT) and stable
across the event, so it is not a consequence of the disturbance. The coils are across the event, so it is not a consequence of the disturbance. The coils are
nominally identical, so this points at orientation relative to a local noise nominally identical, so this points at orientation relative to a local noise
source rather than a sensor fault. X at 84 nT is within ~3× of the 30 nT source rather than a sensor fault. X at 84 nT is within ~3× of the 30 nT
@ -437,12 +437,12 @@ at t ≈ 41 s, so statistics are split around that:
Table 3-5 gives a flat 50 mV<sub>PP</sub> ripple limit on AVDD and DVDD with Table 3-5 gives a flat 50 mV<sub>PP</sub> ripple limit on AVDD and DVDD with
**no frequency qualifier**, and no transfer function from ripple to field error **no frequency qualifier**, and no transfer function from ripple to field error
— no µT-per-mV, no PSRR, no ripple-vs-noise curve. , no µT-per-mV, no PSRR, no ripple-vs-noise curve.
Slow drift should matter less than fast ripple, though this is inference from Slow drift should matter less than fast ripple, though this is inference from
the described architecture rather than an explicit claim. §4.1 describes the the described architecture rather than an explicit claim. §4.1 describes the
output as "the difference in the time to complete the measurement for each output as "the difference in the time to complete the measurement for each
bias" — a differential measurement across forward and reverse coil bias — and §2 bias": a differential measurement across forward and reverse coil bias: and §2
claims measurements are "stable over temperature and inherently free from offset claims measurements are "stable over temperature and inherently free from offset
drift". Anything common to both half-measurements largely cancels, so supply drift". Anything common to both half-measurements largely cancels, so supply
variation slower than one measurement cycle should be rejected far better than variation slower than one measurement cycle should be rejected far better than
@ -453,16 +453,15 @@ the CH347's 3V3 rail ripple actually measures against the 50 mV limit.
--- ---
## Interleaved supply A/B, 2026-08-23 — the scale factor, measured ## Interleaved supply A/B, 2026-08-23, the scale factor, measured
Four captures, LDO / 3V3 / LDO / 3V3 at cycle counts 100, 100, 400, 400, ~6 Four captures, LDO / 3V3 / LDO / 3V3 at cycle counts 100, 100, 400, 400, ~6
minutes each, rails measured at the board: **3.006 V** and **3.353 V**. Full minutes each, rails measured at the board: **3.006 V** and **3.353 V**. Full
write-up, figures and raw output in write-up, figures and raw output in [NOISE_FLOOR.md](NOISE_FLOOR.md); this is
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md); this is
the summary and the things that changed the tooling. the summary and the things that changed the tooling.
**The oscillator moves with the supply.** +4.94% from 3.006 V to 3.353 V, **The oscillator moves with the supply.** +4.94% from 3.006 V to 3.353 V,
measured independently at two cycle counts and agreeing to **0.04%** — V^+0.44. measured independently at two cycle counts and agreeing to **0.04%**, V^+0.44.
This is the cleanest number in the session because, unlike |B|, it cannot be This is the cleanest number in the session because, unlike |B|, it cannot be
contaminated by the sensor moving. contaminated by the sensor moving.
@ -470,7 +469,7 @@ contaminated by the sensor moving.
counts to 0.74%. Exactly ratiometric with the rail would be V^−1; measured is counts to 0.74%. Exactly ratiometric with the rail would be V^−1; measured is
V^−0.65, and correcting by the rail overshoots by +3.5 to +4.3% against a V^−0.65, and correcting by the rail overshoots by +3.5 to +4.3% against a
run-to-run reproducibility of 2.3–3.1%. Correcting by the **measured sample run-to-run reproducibility of 2.3–3.1%. Correcting by the **measured sample
rate** instead leaves −2.6 to −1.9%, inside that floor — and needs nothing rate** instead leaves −2.6 to −1.9%, inside that floor, and needs nothing
external, since the logger already calibrates the period every run. Applying both external, since the logger already calibrates the period every run. Applying both
corrections is clearly wrong (+8.7%): the rate change *is* the supply change seen corrections is clearly wrong (+8.7%): the rate change *is* the supply change seen
through the oscillator, so stacking them counts it twice. through the oscillator, so stacking them counts it twice.
@ -479,8 +478,8 @@ through the oscillator, so stacking them counts it twice.
rotation, so this does not invalidate the above, but translation through a rotation, so this does not invalidate the above, but translation through a
gradient would, and only a clamped re-run can bound that. gradient would, and only a clamped re-run can bound that.
**Cycle count 400 carries an artefact that cycle count 100 does not.** Tones at **Cycle count 400 carries an artifact that cycle count 100 does not.** Tones at
exactly fs/4 and fs/2 of each capture's *own* rate — the two cc=400 runs differ exactly fs/4 and fs/2 of each capture's *own* rate, the two cc=400 runs differ
by 4.9% in rate and each peaks at its own fs/4, so it is locked to the sampling by 4.9% in rate and each peaks at its own fs/4, so it is locked to the sampling
rather than to any frequency. On one axis it reaches 2.43 nT at 25.7σ. Mechanism rather than to any frequency. On one axis it reaches 2.43 nT at 25.7σ. Mechanism
unresolved; it is not a host-side period, since 4 samples is 52.1 ms at one rate unresolved; it is not a host-side period, since 4 samples is 52.1 ms at one rate
@ -492,7 +491,7 @@ fs/4 and fs/2 harmonics. That excludes a fixed digital pattern (which would be
1.000, +2.5σ) and external pickup (0.935, +3.3σ), and sits between V and V², 1.000, +2.5σ) and external pickup (0.935, +3.3σ), and sits between V and V²,
which two rail voltages cannot separate. which two rail voltages cannot separate.
**Every capture drifts through its own thermal warm-up** — 186 to 1241 ppm, **Every capture drifts through its own thermal warm-up**, 186 to 1241 ppm,
monotonically downward, measured by re-fitting the rate on sub-windows monotonically downward, measured by re-fitting the rate on sub-windows
(`calibrate.rate_stability`). That is the error bar under every timing figure (`calibrate.rate_stability`). That is the error bar under every timing figure
here, and the reason the collection-interval mechanism is favoured rather than here, and the reason the collection-interval mechanism is favoured rather than
@ -507,26 +506,26 @@ continuously across the session; this resets at each `logger.py` invocation.
The **field** drift within a run is separate and probably is diurnal: |B| moves The **field** drift within a run is separate and probably is diurnal: |B| moves
−1.4 to −1.8 nT/min in three of the four runs, in a consistent direction, which −1.4 to −1.8 nT/min in three of the four runs, in a consistent direction, which
is the right order for Sq variation. It cannot be the 2.48% residual — that is is the right order for Sq variation. It cannot be the 2.48% residual: that is
1,166 nT against ~40 nT of diurnal across the whole 26-minute session. 1,166 nT against ~40 nT of diurnal across the whole 26-minute session.
**The sensor moved during a run, not only between them.** LDO/cc400 rotates **The sensor moved during a run, not only between them.** LDO/cc400 rotates
**0.49°** across its own run, ten times the quietest, and that is the same run **0.49°** across its own run, ten times the quietest, and that is the same run
carrying 485 nT of Z drift at nearly constant |B| — which is rotation, not field. carrying 485 nT of Z drift at nearly constant |B|, which is rotation, not field.
The rig was not restrained and had to be handled to move the 3 V jumper between The rig was not restrained and had to be handled to move the 3 V jumper between
runs, so this is expected rather than surprising; it is why the residual is runs, so this is expected rather than surprising; it is why the residual is
attributed to movement. attributed to movement.
**Filtering and decimating a fast capture matches sampling slowly**, which the **Filtering and decimating a fast capture matches sampling slowly**, which the
README had been asserting without evidence. Decimating a cc=100 capture by 4 README had been asserting without evidence. Decimating a cc=100 capture by 4
changes its own broadband floor by **+0.3 to +0.5%** — a comparison with no changes its own broadband floor by **+0.3 to +0.5%**: a comparison with no
second run in it, so no gain or position difference to confound it. It is also second run in it, so no gain or position difference to confound it. It is also
strictly better: the decimated path carries none of the fs/4 artefact, and a strictly better: the decimated path carries none of the fs/4 artifact, and a
proper anti-alias filter removes 60 Hz before it can fold, which sampling slowly proper anti-alias filter removes 60 Hz before it can fold, which sampling slowly
cannot. cannot.
**The dither question is settled at cycle count 100.** 0.65–0.81 LSB per sample, **The dither question is settled at cycle count 100.** 0.65–0.81 LSB per sample,
and averaging 1,024 samples reaches 0.035–0.050 LSB — within 1.5–2.1× of ideal. and averaging 1,024 samples reaches 0.035–0.050 LSB, within 1.5–2.1× of ideal.
A comb in the residual histogram is not evidence against this: the residual of an A comb in the residual histogram is not evidence against this: the residual of an
integer-valued signal is a comb at one LSB by construction, and its sharpness integer-valued signal is a comb at one LSB by construction, and its sharpness
tracks only how far that axis drifted. tracks only how far that axis drifted.
@ -547,21 +546,197 @@ would have biased a supply comparison:
--- ---
## Host setup: why the udev rule looks the way it does
Moved here from the README, which now states the rule and not its history.
The CH347's USB node defaults to `root:root 0664`, so it needs a rule. The one
`setup.sh` installs grants access two ways, because no single mechanism covers
every distro:
```
# /etc/udev/rules.d/60-ch347.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", TAG+="uaccess"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", GROUP="plugdev", MODE="0660"
```
`uaccess` has systemd-logind put an ACL on the node for whoever holds the local
seat, no group, no logout, and is the only mechanism that can work on atomic
Fedora. `GROUP`/`MODE` is the fallback for ssh sessions and seatless systems.
**Two details are load-bearing**, and getting either wrong produces a rule that
installs cleanly and grants nothing:
- **The prefix must sort below 73.** udev merges every rules directory into one
lexicographic sequence, and the only thing that acts on the tag is
`TAG=="uaccess", ENV{MAJOR}!="", RUN{builtin}+="uaccess"` in systemd's
`73-seat-late.rules`. At `99-` the tag is added after that line has already
run: set, never read, no ACL. systemd keeps its own uaccess rules in
`70-uaccess.rules` for this reason.
- **`TAG+="uaccess"` and `GROUP=` must be on separate lines.** udev discards a
whole rule line whose `GROUP=` it cannot resolve, and since systemd 258 that
includes any group that exists but is not a *system* group. Sharing a line
means an unusable group silently takes the uaccess tag down with it.
Together those explain a failure that looks distro-specific but is not: on
Debian and Ubuntu the `plugdev` group grants access on its own, masking a
uaccess tag that never fired. Fedora removed `plugdev` years ago, so on
**Bazzite** neither half worked and the adapter stayed inaccessible.
Two more atomic-Fedora traps, which `setup.sh` prints fixes for:
- `usermod -aG dialout $USER` fails with *"group 'dialout' does not exist"* even
though `getent` finds it, because `nss_altfiles` lets `getent` read
`/usr/lib/group` while `usermod` writes `/etc/group` alone. Copy the line
across first, then re-run `usermod` and log back in.
- Bazzite is known not to reload `/etc/udev/rules.d` when it switches to the
final rootfs ([ublue-os/bazzite#2516](https://github.com/ublue-os/bazzite/issues/2516)),
so a correct rule can sit inert until
`sudo udevadm control --reload-rules && sudo udevadm trigger` is run once.
`setup.sh` checks the node itself at the end rather than guessing, and says
*which* mechanism granted access, `uaccess ACL` or `group membership`, since
the two fail in different ways and only one is available on Bazzite.
---
## Design rationale, moved out of the README
The README states what the code does; this is why it does it that way.
### How the time base and miss detection work
**The chip samples on its own internal schedule**, unaffected by bus traffic
(§5.8.2, §5.7.2). Two consequences drive the design: read jitter is not sample
jitter, so `sample_index` is a grid coordinate; and a late read returns the
*newer* measurement, so an unnoticed miss would silently compress the time axis.
A lost measurement is therefore **recorded, not dropped**: a placeholder row
keeps `sample_index` contiguous and makes the gap explicit. Double-counting is
structurally impossible: `HSHAKE` DRC1=1 makes DRDY clear only on a results read,
and `configure()` verifies the readback rather than assuming it. If the writer
queue ever fills the run **stops** rather than dropping a row, since a hole in
`sample_index` is worse than a short capture.
**The period is calibrated before recording starts**, because counting how many
grid points passed unseen needs the real period and the TMRC table value is 6–9%
out on this unit. `calibrate_period()` fits one second of loss-free samples to
~0.07%, then keeps refining from clean intervals to follow thermal drift. If no
clean stretch can be found the run aborts before recording: that is the honest
signal that the requested rate is not sustainable.
**Counting lost measurements uses two signals**, because the two questions have
different best answers. *Did we lose any?*, the **DRDY bracket**, which is exact
where the read-to-read interval is only a heuristic (measured here the interval
overshoots a 28.8 ms period by 29% on host stalls alone, while the bracket stays
under 10 ms). *How many?*: the **interval since the previous accepted sample**,
rounded to whole periods, since every accepted read sits on a grid point.
**Clock drift is compensated in post-processing, not baked into the file.**
`capture.py` regresses the host clock on `sample_index` to recover the true
period: short-term regularity from the chip, long-term rate calibration from the
host. Over hours no single slope fits; `capture.py` reports that as
`drift_limited` and says to analyse shorter windows via `Capture.restrict()`.
**Threading.** The sampling loop does I²C and nothing else, pushing raw counts
onto a bounded queue; a writer thread does unit conversion, CSV formatting,
flushing and the console display. The sampler releases the GIL inside each USB
transfer and must re-acquire it, so Python's default 5 ms switch interval becomes
the jitter floor, `sys.setswitchinterval(0.0005)` cut the worst bracket from
9.96 ms to 5.65 ms. The writer is reniced out of the way, which needs no
privilege; `--high-priority` additionally raises the sampler, which does.
### Choosing a cycle count
*dither* is the sensor's own noise in LSB. Below roughly 0.2 LSB the quantiser
stops being dithered and averaging no longer recovers sub-LSB resolution. At
cc=100 this is **measured, not assumed**: dither came out at 0.65–0.81 LSB and
averaging 1,024 samples reached 0.035–0.050 LSB, within 1.5–2.1× of the ideal
`1/√n`. `characterize.py` prints this for any capture; fall back to cycle count
200 if its dither check shows the LSB column flattening rather than falling.
**The default is cycle count 100 at 750 kHz**, which runs the sensor at its own
~283 Hz ceiling. It costs 1.5% in post-filter noise against cc=200 and buys 1.9×
the spectrum, which is worth taking because **aliased interference cannot be
filtered out afterwards at any cycle count**, at 283 Hz both mains and its
second harmonic are in band and can be notched. cc=50 is right when something
above 141 Hz needs identifying, but its dither margin is the thinnest here.
**Prefer decimating a fast capture over sampling slowly.** Decimation changes a
capture's own broadband floor by **+0.3 to +0.5%**, so the two are equivalent for
noise, and decimation is strictly better on everything else, since a natively
slow capture folds 60 Hz irrecoverably and carries sample-locked lines the
decimated path does not have at all. `compare.py` prints this comparison whenever
two captures differ by an integer cycle-count factor.
### CH347 access
The adapter presents three USB interfaces. Interfaces 0 and 1 are CDC-ACM and the
kernel binds them as `/dev/ttyACM0` (the UART). **Interface 2** is vendor-class,
carries I2C/SPI/JTAG with no kernel driver bound, and libusb can claim it
directly. `ch347.py` deliberately never calls `set_configuration()`: the device
is already configured, and re-setting it would disturb the CDC-ACM interfaces.
There is no usable off-the-shelf option: no CH347 kernel driver exists (the
in-tree `spi-ch341` is for `1a86:5512`, a different chip), the PyPI `ch347`
package wraps a Windows DLL, and `ch347api` supports only HID mode (`55dc`), not
this vendor-bulk Mode 1. The framing follows the `aystarik/ch347-i2c-spi-gpio`
Linux driver, whose id-table entry
`USB_DEVICE_INTERFACE_NUMBER(0x1a86, 0x55db, 0x02)` matches this device exactly.
Wire format, bulk OUT `0x06`, bulk IN `0x86`, max 63 bytes per transfer:
| Purpose | Bytes out | Bytes back |
|---|---|---|
| Set speed | `AA 6<speed> 00` | 0 |
| Write n | `AA 74 (80\|n+1) (addr<<1) <data> 75 00` | n+1 |
| Read n | `AA 74 81 ((addr<<1)\|1) [C0\|(n-1)] C0 75 00` | n+1 |
| Write n then read m | `AA 74 (80\|n+1) (addr<<1) <data> 74 81 ((addr<<1)\|1) [C0\|(m-1)] C0 75 00` | n+2+m |
| Probe | `AA 74 81 (addr<<1) 75 00` | 1 |
**Every returned byte must be `1`**: that is the per-byte ACK. On a read, the
leading bytes are address ACKs and the rest is payload. `tests/test_ch347.py`
pins all five forms byte for byte.
### RM3100 quirks
Each of these is a place where the part or the manual is not what you would
guess; `rm3100.py` carries the detail at the constant that encodes it.
- **Register reads use the plain address, not `|0x80`.** §5 describes the SPI
convention of adding 0x80, but §5.8.4's I2C diagram writes `0x24` literally.
- **A register read uses a repeated START**, not the STOP-then-START the manual
draws (§4.5.2, §5.8.4). The part accepts it and it halves the USB round trips,
which is what sets the sample-rate ceiling. `rm3100.read_reg()` falls back to
the manual's form on a bus that cannot do it.
- **CMM = 0x79** for all three axes. Table 5-1 calls bit 3 reserved-zero, but the
manual's own examples (§5.7.2, §5.8.3) set it. The examples win.
- **HSHAKE is set to 0x0A** (`DRC0=0`, `DRC1=1`) during init. The 0x1B default has
`DRC0=1`, meaning *any* register write clears DRDY, including the pointer write
that reading STATUS itself requires, so polling could never observe DRDY set.
- **Gain** is `0.3671 × cycle_count + 1.5` LSB/µT, a linear fit to Table 3-1.
- **The rate model divisor is specified, not fitted:** a 180 kHz circuit
oscillation (Table 3-1) over two bias directions (§4.1) gives 90,000 counts/s.
- Measurements are three **24-bit big-endian two's-complement** values, read as
9 bytes from 0x24 using the sensor's register auto-increment.
---
## Open questions ## Open questions
- Re-run the LDO comparison with the sensor **clamped**, and in an order that is - Re-run the LDO comparison with the sensor **clamped**, and in an order that is
not degenerate. The 2026-08-23 session used L,3,L,3 against cycle counts not degenerate. The 2026-08-23 session used L,3,L,3 against cycle counts
100,100,400,400, which makes `t = supply + 2·cc` exactly — supply, cycle count 100,100,400,400, which makes `t = supply + 2·cc` exactly: supply, cycle count
and a linear drift cannot all three be fitted. L,3,3,L breaks it. and a linear drift cannot all three be fitted. L,3,3,L breaks it.
- Identify the **fs/4 artefact** at cycle count 400. A cc=200 run at both - Identify the **fs/4 artifact** at cycle count 400. A cc=200 run at both
supplies would say whether it scales with cycle count or appears at a threshold. supplies would say whether it scales with cycle count or appears at a threshold.
- Fix `rm3100.AXIS_OVERHEAD_S`: 68.7 µs against 38–41 µs measured on this unit, - Fix `rm3100.AXIS_OVERHEAD_S`: 68.7 µs against 38–41 µs measured on this unit,
which puts the cc100:cc400 rate ratio 1.9% out. which puts the cc100:cc400 rate ratio 1.9% out.
- Compare |B| against IGRF/WMM for the capture site (39.661262° N, - Compare |B| against IGRF/WMM for the capture site (39.661262° N,
82.604173° W, 255 m). Not a calibration standard — indoor hard and soft iron 82.604173° W, 255 m). Not a calibration standard, indoor hard and soft iron
are unknown and nothing here is corrected — but it is the only absolute are unknown and nothing here is corrected: but it is the only absolute
reference available, and it would say which rail reads closer to right. reference available, and it would say which rail reads closer to right.
- Deal with 60 Hz coupling at source — standoff distance, then a repeat - Deal with 60 Hz coupling at source: standoff distance, then a repeat
characterisation — rather than only sampling fast enough to see it. characterisation, rather than only sampling fast enough to see it.
- Reliability over extended runs on bare metal, where the VM ceilings above do - Reliability over extended runs on bare metal, where the VM ceilings above do
not apply. not apply.

588
README.md
View file

@ -9,8 +9,11 @@ Reference throughout: *RM3100 & RM2100 Sensor Suite User Manual*, Doc 1017252
R07 (`PNI Sensor - RM3100-Sensor-Suite-User-Manual-R07-1.pdf`, in this R07 (`PNI Sensor - RM3100-Sensor-Suite-User-Manual-R07-1.pdf`, in this
directory). Section numbers below refer to it. directory). Section numbers below refer to it.
[NOTES.md](NOTES.md) holds the measurement notebook — what was measured on this This file documents how the code works *now*. How that was arrived at, every
rig, and why the design ended up as it is. measurement plus a table of conclusions later measurements overturned, is in
[Noise_Floor_Testing/NOTES.md](Noise_Floor_Testing/NOTES.md), with the supply
and rate-scaling analysis in
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
## Quickstart ## Quickstart
@ -19,12 +22,7 @@ rig, and why the design ended up as it is.
./.venv/bin/python diagnose-comms.py # walk the signal chain, stop at the first fault ./.venv/bin/python diagnose-comms.py # walk the signal chain, stop at the first fault
./.venv/bin/python logger.py --duration 60 # capture to rm3100_<timestamp>.csv ./.venv/bin/python logger.py --duration 60 # capture to rm3100_<timestamp>.csv
./.venv/bin/python characterize.py rm3100_*.csv # noise floor, spectrum, stability ./.venv/bin/python characterize.py rm3100_*.csv # noise floor, spectrum, stability
``` ./.venv/bin/python -m pytest # the suite needs no hardware
The test suite needs no hardware:
```bash
./.venv/bin/python -m pytest
``` ```
## Hardware ## Hardware
@ -35,43 +33,36 @@ The RM3100 is on a breakout board. All connections below are confirmed working.
| Adapter | RM3100 | Notes | | Adapter | RM3100 | Notes |
|---|---|---| |---|---|---|
| I2C VCC (3V3) | DVDD | digital supply | | I2C VCC (3V3) | DVDD | digital supply |
| I2C VCC (3V3) | VDD (= AVDD) | analog supply — **required for measurements** | | I2C VCC (3V3) | VDD (= AVDD) | analog supply, **required for measurements** |
| I2C VCC (3V3) | I2CEN | **required to select I2C over SPI** | | I2C VCC (3V3) | I2CEN | **required to select I2C over SPI** |
| I2C GND | DVSS | digital ground | | I2C GND | DVSS | digital ground |
| I2C SCL | SCK/SCL | pin 27, shared with SPI SCLK | | I2C SCL | SCK/SCL | pin 27, shared with SPI SCLK |
| I2C SDA | SI/SDA | pin 1, shared with SPI MOSI | | I2C SDA | SI/SDA | pin 1, shared with SPI MOSI |
All three 3V3 connections share the adapter's single I2C VCC pin. Current draw is All three 3V3 connections share the adapter's single I2C VCC pin. Current draw is
negligible — ~260 µA per axis at 24 Hz (Table 3-1). negligible, ~260 µA per axis at 24 Hz (Table 3-1).
Four pins are worth knowing about; two of them cause failures that are easy to Four pins matter; two of them cause failures that are easy to misdiagnose.
misdiagnose.
**I2CEN (pin 22) must be tied HIGH.** `HIGH = I2C, LOW = SPI` (§4.3.1). SDA/SCL - **I2CEN (pin 22) must be tied HIGH.** `HIGH = I2C, LOW = SPI` (§4.3.1), and
are shared with SPI MOSI/SCLK and I2CEN selects between them. Left floating or SDA/SCL are shared with SPI MOSI/SCLK. Left floating or low the chip stays in
low, the chip stays in SPI mode and **never ACKs its I2C address** — the bus scan SPI mode and **never ACKs its I2C address**, so the bus scan finds nothing at
finds nothing at all, even though the adapter is perfectly healthy. At all on a perfectly healthy adapter. At DVDD = 3.3 V it needs
DVDD = 3.3 V it needs `VIH = 0.7 × DVDD` = 2.31 V, so tie it directly to 3V3. `VIH = 0.7 × DVDD` = 2.31 V, so tie it directly to 3V3.
- **AVDD/AVSS (pins 4, 5) must be powered to take measurements.** With only DVDD
**AVDD/AVSS (pins 4, 5) must be powered to take measurements.** §4.3.1: *"AVDD the part is **half alive** in a way that reads as working: the address
can be turned off when not making a measurement to conserve power, since all responds, REVID reads `0x22`, registers write and read back, while the analog
other operations are supported with DVDD."* With only DVDD connected the part is section is dead, DRDY never asserts and the measurement registers stay zero.
**half alive** in a way that reads as working: the address responds, REVID reads BIST reporting `XOK=YOK=ZOK=0` is the unambiguous tell. DVDD must come up
`0x22`, registers write and read back — but the analog section driving the coils **before or with** AVDD, never after, and the two must stay within **0.1 V**
is dead, so DRDY never asserts and the measurement registers stay at zero. BIST while running (§4.3.1, Table 3-5), which one shared 3V3 rail gives for free.
reporting `XOK=YOK=ZOK=0` is the unambiguous tell. Both rails are needed, and - **SA0 (pin 3) / SA1 (pin 28) set the address**, top five bits fixed at
AVDD is not a substitute for DVDD: DVDD must come up **before or with** AVDD, `0b01000`, giving 0x20–0x23 (§4.5). On this breakout both straps are high, so
never after, and the two must stay within **0.1 V** while running (§4.3.1, the sensor answers at **0x23**. `logger.py` scans all four, so no configuration
Table 3-5) — which sharing one 3V3 rail satisfies for free. is needed. The pins are shared with SPI SSN and MISO, so a breakout may label
them `SSN/SA0` and `MISO/SA1`.
**SA0 (pin 3) / SA1 (pin 28) set the address.** The top five bits are fixed at - **DRDY (pin 23) is not connected.** The driver polls the STATUS register,
`0b01000`, giving 0x20–0x23 (§4.5). On this breakout both straps are high, so the which the manual explicitly offers as an alternative (§5.4.1).
sensor answers at **0x23**. The pins are shared with SPI SSN and MISO, so a
breakout may label them `SSN/SA0` and `MISO/SA1`. `logger.py` scans all four
addresses, so no configuration is needed.
**DRDY (pin 23) is not connected.** Not required — the driver polls the STATUS
register instead, which the manual explicitly offers as an alternative (§5.4.1).
### Power ### Power
@ -84,37 +75,18 @@ register instead, which the manual explicitly offers as an alternative (§5.4.1)
Reference decoupling (Figures 4-1, 4-2) is a **10 µF bulk capacitor in parallel Reference decoupling (Figures 4-1, 4-2) is a **10 µF bulk capacitor in parallel
with 0.1 µF ceramic**, rail to ground, feeding AVDD, both DVDD pins and I2CEN. with 0.1 µF ceramic**, rail to ground, feeding AVDD, both DVDD pins and I2CEN.
That is the entire filtering specification — no ferrite, no split rails, no LDO That is the entire filtering specification: no ferrite, no split rails, no LDO
requirement. §4.2.3 adds a placement rule: *"Keep capacitors, especially tantalum requirement. §4.2.3 adds a placement rule: *"Keep capacitors, especially tantalum
capacitors, far away from the sensor coils"* — a magnetic concern, not an capacitors, far away from the sensor coils"*, which is a magnetic concern.
electrical one.
**Every number in Table 3-1 is quoted at 3.0 V** (footnote 1: *"performance will **Every number in Table 3-1 is quoted at 3.0 V** (footnote 1: *"performance will
differ"*). 3.353 V is comfortably in range, but the gain fit behind differ"*), so the gain fit behind `tesla_per_count` is strictly a 3.0 V figure.
`tesla_per_count` is strictly a 3.0 V figure, so running there is an Measured on interleaved captures at 3.006 V and 3.353 V, +0.347 V moves the
extrapolation off the calibration point. **This has now been measured**, on sample rate by **+4.94%** and |B| by **−7.1%**, with no resolvable change in
interleaved captures at 3.006 V and 3.353 V: broadband noise. So **supply quality does not set the noise floor, but supply
*voltage* sets the scale**, at roughly −20 %/V. Both effects are one thing seen
| | effect of +0.347 V | twice, and the calibrated period every capture already records is a proxy for the
|---|---| rail; see [Calibration](#calibration).
| **Sample rate** | **+4.94%** (repeats to 0.04% across two cycle counts) |
| **Total field \|B\|** | **−7.1%**, about −3,245 nT |
| Broadband noise | none resolvable, ~4% fractionally |
So **supply quality does not set the noise floor, but supply *voltage* sets the
scale**, at roughly −20 %/V. A ±1% regulator at 3.0 V holds scale to ~0.2%; a
USB-derived rail free to move ±5% holds it to ~1%, on top of the ~7% fixed offset
from sitting off the calibration point.
Both effects are the same thing seen twice. The chip's measurement interval is
clocked by its own oscillator — the per-axis overhead is a fixed ~3.6 *counts*,
not a fixed time — so the same number of counts is collected over a window whose
duration goes as 1/f. Gain follows integration time, giving `gain ∝ (cc + n)/f`
with **no free exponent**. That makes the calibrated period every capture already
records a proxy for the rail, correctable without a voltmeter. Full analysis,
including what is *not* settled, in
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
[NOTES.md](NOTES.md) covers what the manual does and does not say about ripple.
## Setup ## Setup
@ -125,7 +97,8 @@ including what is *not* settled, in
Idempotent. It checks the adapter is present, installs a udev rule, creates Idempotent. It checks the adapter is present, installs a udev rule, creates
`.venv` with pyusb, numpy, matplotlib and pytest, then verifies the device node `.venv` with pyusb, numpy, matplotlib and pytest, then verifies the device node
is actually writable and prints whichever fix applies if it is not. `sudo` is is actually writable and prints whichever fix applies if it is not. `sudo` is
needed for the udev rule and nothing else. needed for the udev rule and nothing else. Python here is PEP 668
externally-managed, hence the venv.
The rule is needed because the CH347's USB node defaults to `root:root 0664`. It The rule is needed because the CH347's USB node defaults to `root:root 0664`. It
grants access two ways, because no single mechanism covers every distro: grants access two ways, because no single mechanism covers every distro:
@ -136,47 +109,12 @@ SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", TAG+="uacce
SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", GROUP="plugdev", MODE="0660" SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", GROUP="plugdev", MODE="0660"
``` ```
`uaccess` has systemd-logind put an ACL on the node for whoever holds the local The `60-` prefix, the two separate lines and the choice of group are all
seat — no group, no logout — and is the only mechanism that can work on atomic load-bearing; getting any of them wrong installs cleanly and grants nothing.
Fedora. `GROUP`/`MODE` is the fallback for ssh sessions and seatless systems, [NOTES.md](Noise_Floor_Testing/NOTES.md) explains why, and covers the
using whichever of `plugdev` or `dialout` exists as a system group. atomic-Fedora traps `setup.sh` prints fixes for. `setup.sh` checks the node
itself at the end and says *which* mechanism granted access: `uaccess ACL` or
**Two details in that file are load-bearing**, and getting either wrong produces `group membership`.
a rule that installs cleanly and grants nothing:
- **The prefix must sort below 73.** udev merges every rules directory into one
lexicographic sequence, and the only thing that acts on the tag is
`TAG=="uaccess", ENV{MAJOR}!="", RUN{builtin}+="uaccess"` in systemd's
`73-seat-late.rules`. At `99-` the tag is added after that line has already
run: set, never read, no ACL. systemd keeps its own uaccess rules in
`70-uaccess.rules` for this reason.
- **`TAG+="uaccess"` and `GROUP=` must be on separate lines.** udev discards a
whole rule line whose `GROUP=` it cannot resolve, and since systemd 258 that
includes any group that exists but is not a *system* group. Sharing a line
means an unusable group silently takes the uaccess tag down with it.
Together those explain a failure that looks distro-specific but is not: on
Debian and Ubuntu the `plugdev` group grants access on its own, masking a
uaccess tag that never fired. Fedora removed `plugdev` years ago, so on
**Bazzite** neither half worked and the adapter stayed inaccessible.
Two more atomic-Fedora traps `setup.sh` prints fixes for:
- `usermod -aG dialout $USER` fails with *"group 'dialout' does not exist"* even
though `getent` finds it, because `nss_altfiles` lets `getent` read
`/usr/lib/group` while `usermod` writes `/etc/group` alone. Copy the line
across first, then re-run `usermod` and log back in.
- Bazzite is known not to reload `/etc/udev/rules.d` when it switches to the
final rootfs ([ublue-os/bazzite#2516](https://github.com/ublue-os/bazzite/issues/2516)),
so a correct rule can sit inert until
`sudo udevadm control --reload-rules && sudo udevadm trigger` is run once.
`setup.sh` checks the node itself at the end rather than guessing, and says
*which* mechanism granted access — `uaccess ACL` or `group membership` — since
the two fail in different ways and only one of them is available on Bazzite.
Python here is PEP 668 externally-managed, hence the venv rather than a
system-wide `pip install`.
## Usage ## Usage
@ -188,31 +126,32 @@ system-wide `pip install`.
| Flag | Default | Meaning | | Flag | Default | Meaning |
|---|---|---| |---|---|---|
| `--rate` | — | target Hz; derives the cycle count and TMRC. Mutually exclusive with `--cycle-count` | | `--rate` | none | target Hz; derives the cycle count and TMRC. Mutually exclusive with `--cycle-count` |
| `--cycle-count` | 100 | sets both the rate and the LSB — see [Configuration](#configuration) | | `--cycle-count` | 100 | sets both the rate and the LSB; see [Configuration](#configuration) |
| `--tmrc` | `0x92` | rate register. The default is the fastest, letting the cycle count set the rate; give one only to sample *slower* | | `--tmrc` | `0x92` | rate register. The default is the fastest, letting the cycle count set the rate; give one only to sample *slower* |
| `--duration` | 0 | seconds, 0 = until Ctrl-C | | `--duration` | 0 | seconds, 0 = until Ctrl-C |
| `--output` | timestamped | CSV path | | `--output` | timestamped | CSV path |
| `--address` | autodetect | skip the bus scan | | `--address` | autodetect | skip the bus scan |
| `--bus-speed` | 750 | I²C kHz. 100 kHz would spend 42% of each period on the bus at the default cycle count; 750 spends 6% | | `--bus-speed` | 750 | I²C kHz. 100 kHz would spend 42% of each period on the bus at the default cycle count; 750 spends 6% |
| `--calibrate` | 1.0 | seconds of loss-free samples used to measure the true period before recording. The run aborts if no clean stretch can be found | | `--calibrate` | 1.0 | seconds of loss-free samples used to measure the true period before recording. The run aborts if no clean stretch can be found |
| `--note` | — | free text recorded in the capture header, e.g. the supply under test | | `--note` | none | free text recorded in the capture header, e.g. the supply under test |
| `--high-priority` | off | raise the sampling thread to nice −10. Needs `CAP_SYS_NICE`, so run under `sudo`; output files are handed back to the invoking user | | `--high-priority` | off | raise the sampling thread to nice −10. Needs `CAP_SYS_NICE`, so run under `sudo`; output files are handed back to the invoking user |
| `--scan-only` | — | scan and exit | | `--scan-only` | none | scan and exit |
**Exit status is non-zero if the capture is compromised in any way** — a lost **Exit status is non-zero if the capture is compromised in any way**: a lost
measurement, an ambiguous gap, a truncated run, or an undrained writer. Each measurement, an ambiguous gap, a truncated run, or an undrained writer. Each
prints an explanation on stderr, and the file is always valid as far as it goes. prints an explanation on stderr, and the file is always valid as far as it goes.
A capture is analysable whether the run ended on `--duration`, on **Ctrl-C**, or
early.
Before recording, `logger.py` prints how the configuration was derived, so it can Before recording, `logger.py` prints how the configuration was derived, so it
be checked rather than trusted, then measures the true sample period against the can be checked, then measures the true sample period against the host clock.
host clock. Both appear in the capture header. Both appear in the capture header.
### Analysis tools ### Analysis tools
```bash ```bash
./.venv/bin/python plot.py capture.csv # -> capture.png: X, Y, Z, |B| ./.venv/bin/python plot.py capture.csv # -> capture.png: X, Y, Z, |B|
./.venv/bin/python plot.py capture.csv --smooth 0 # raw trace only
./.venv/bin/python characterize.py capture.csv # -> capture_noise.png + summary ./.venv/bin/python characterize.py capture.csv # -> capture_noise.png + summary
./.venv/bin/python characterize.py capture.csv --trim 30 # drop settling/handling ./.venv/bin/python characterize.py capture.csv --trim 30 # drop settling/handling
./.venv/bin/python compare.py --group note *.csv # A/B two conditions ./.venv/bin/python compare.py --group note *.csv # A/B two conditions
@ -221,60 +160,40 @@ host clock. Both appear in the capture header.
./.venv/bin/python sweep.py --rates 10,50,150 # measure what each rate delivers ./.venv/bin/python sweep.py --rates 10,50,150 # measure what each rate delivers
``` ```
- **`plot.py`** — four stacked panels with independent y-scales, since the three Each tool's `--help` carries its full reasoning; in one line each:
axes sit at very different DC offsets.
- **`characterize.py`** — amplitude spectral density against the 1.2 nT/√Hz of
Table 3-1, Allan deviation, residual distribution, and host read latency, all
on **fixed axes** so two runs can be laid side by side. Reports `white sd`
(`sd(diff)/√2`, which rejects drift) beside the plain `sd`, and a **dither
check** — whether averaging still gets below one quantiser step. Flags any line
sitting at a simple fraction of the sample rate. Also writes a
`_spectrogram.png`: |B| in time and frequency at 0.025 Hz resolution, with
fs/4 and mains marked — where mains has folded, the line is drawn at the alias
and labelled as one. Drawn in **ppm of |B| per √Hz** on a fixed colour scale,
with native bins averaged to a fixed 0.1 Hz step so every capture gets the
same degrees of freedom per cell; both are needed before one scale across
several captures means anything. `--spectrogram`, `--overlap`, `--max-freq`
and `--colormap` tune it; the default `turbo` makes narrow lines legible,
`viridis` is the lightness-monotonic option.
Two further figures come out beside it:
`_drift.png` — the chip's oscillator measured against the host clock, as rate
against elapsed time with error bars, and beside it the two-sample deviation
σ_y(τ) that supplies those bars. The minimum of that curve is the best the
rate can be known and the averaging time worth using; the branch rising ∝ τ to
its right is real drift, not measurement noise. These parts run 186–1241 ppm
of warm-up against a floor near 10 ppm at τ ≈ 2–10 s.
`_timebase.png` — for each detected line, its coherent amplitude computed on
the chip's sample grid and again on the host clock. A line at a fixed
frequency sharpens on the host clock; one locked to the sampling is wrecked by
it. That settles **from a single capture** what `compare.py` otherwise needs
two different rates to decide. Diagnostic only: no other figure uses the host
time base, because neither choice is right for both kinds of line.
- **`compare.py`** — everything cross-capture. Reports noise **fractionally**
(ppm of |B|), because a gain change carries the noise with it and an absolute
comparison reads a pure scale change as a noise difference. Also: one common
band for every capture; the timing model solved per condition; whether a line
is locked to the sampling or to a frequency; whether filtering and decimating a
fast capture matches a natively slow one; and |B| under each candidate
correction. Label captures `CONDITION/variant=path` — the part before the `/`
is the thing under test, and `--supply CONDITION=VOLTS` attaches a rail to it.
- **`sweep.py`** — runs `logger.py` once per target rate and tabulates what each
configuration actually achieved. `--calibration-out` fits the timing model
across the sweep and writes it as a calibration.
- **`calibrate.py`** — converts a raw capture to a nanotesla CSV, applying a
calibration when one is named and nothing when one is not. See below.
`--trim SECONDS` drops that much from *both* ends of a capture. A run usually - **`plot.py`**: four stacked panels with independent y-scales, since the three
opens while the sensor is still settling and closes with a hand on the rig, and axes sit at very different DC offsets.
neither end is a noise measurement. Nothing is dropped unless asked, and a - **`characterize.py`**: spectral density against Table 3-1, Allan deviation,
capture shorter than 4× the trim is analysed whole with a note rather than gutted. residual distribution and host read latency on **fixed axes**, plus a dither
check and any line at a simple fraction of the sample rate. Writes three more
figures beside it: `_spectrogram.png` (whether a feature lasted or was a
burst), `_drift.png` (the chip's oscillator against the host clock, with
σ_y(τ) supplying the error bars) and `_timebase.png` (which clock each line is
coherent on, which settles from one capture what `compare.py` needs two rates
to decide).
- **`compare.py`**: everything cross-capture: fractional noise in ppm of |B|,
one common band for every capture, the timing model per condition, whether a
line is locked to the sampling or to a frequency, whether decimating a fast
capture matches a natively slow one, and |B| under each candidate correction.
Label captures `CONDITION/variant=path`; `--supply CONDITION=VOLTS` attaches a
rail.
- **`sweep.py`**: runs `logger.py` once per target rate and tabulates what each
configuration achieved. `--calibration-out` fits the timing model across the
sweep and writes it as a calibration.
- **`calibrate.py`**: converts a raw capture to a nanotesla CSV, applying a
calibration when one is named and nothing when one is not.
`--trim SECONDS` drops that much from *both* ends of a capture, since a run
usually opens while the sensor is settling and closes with a hand on the rig. Nothing is
dropped unless asked, and a capture shorter than 4× the trim is analysed whole
with a note.
### Calibration ### Calibration
**Nothing is corrected by default.** `calibrate.py` without `--calibration` **Nothing is corrected by default.** `calibrate.py` without `--calibration`
changes units and nothing else, and the analysis tools read raw captures as they changes units and nothing else, and the analysis tools read raw captures as they
always did. That is deliberate: the corrections below are real but not yet always did. The corrections are real but not settled enough to apply silently.
settled enough to apply silently.
```bash ```bash
./.venv/bin/python sweep.py --rates 32,64,128,256 \ ./.venv/bin/python sweep.py --rates 32,64,128,256 \
@ -285,49 +204,32 @@ settled enough to apply silently.
A calibration is **per unit and per supply**, because the chip times everything A calibration is **per unit and per supply**, because the chip times everything
from one oscillator and that oscillator is neither the specified frequency nor from one oscillator and that oscillator is neither the specified frequency nor
fixed. Solving `period = 3 × (cc/C + overhead)` freely against one unit gave fixed. It is worth having even if you never correct a field, because it makes the
C = 88,546 with a 40.6 µs overhead at 3.006 V, against the nominal 90,000 and timing model right: with a measured pair the predicted rate matches the measured
68.7 µs — and C moved to 92,889 at 3.353 V. one at both cycle counts, where the nominal model is 1.9% out between them. Two
corrections then become available, neither on by default:
That is worth having even if you never correct a field, because it makes the | | what it is |
timing model right: with the measured pair, the predicted rate matches the |---|---|
measured one to −0.00% at both cycle counts, where the nominal model is 1.9% out | oscillator gain | `(C / reference) ^ gain_exponent`, where the exponent is **1**, which an oscillator-clocked measurement interval predicts |
between them. Once it is right, **one capture recovers the oscillator** — | cycle-count gain | `gain ∝ (cc + gain_offset_counts)`; Table 3-1 implies n = 4.086, measurement puts it near zero |
`C = cc / (period/3 − overhead)` — to about 0.09% at cycle count 100.
Two corrections then become available, and neither is on by default: Every correction arrives with a propagated standard deviation, from the timing
fit and from `rate_stability()`, which measures how far the oscillator drifted
during a run and reads **186 to 1241 ppm** here. Let the rig settle first.
| | what it is | status | The output is a different format and `capture.py` refuses to read it, which
|---|---|---| keeps raw captures the one source of truth. Its header carries the source's
| oscillator gain | `(C / reference) ^ gain_exponent` | −7.1% for +0.347 V. The exponent is **1**, which an oscillator-clocked measurement interval predicts and the measured rates support at 0.3σ against 1.2σ for the alternative | SHA-256, so a derived file can be tied back to what produced it.
| cycle-count gain | `gain_offset_counts` | `gain ∝ (cc + n)`. Table 3-1 implies n = 4.086; measured on one unit it is 0.90 and −0.09. Together the two corrections take a 10.3% disagreement to 2.5%, against a 2.3–3.1% movement floor |
Every correction arrives with a propagated standard deviation — 0.3–0.4% on
these captures, well under the residual, which is what says the residual is real
rather than calibration slop. It comes from the timing fit and from
`rate_stability()`, which measures how far the oscillator drifted during a run:
**186 to 1241 ppm** here, monotonically downward. Let the rig settle first.
The output is a different format and `capture.py` will refuse to read it, on
purpose — raw captures stay the one source of truth. Its header carries the
source's SHA-256 so a derived file that has drifted from its source is
detectable rather than merely plausible. Full analysis in
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
> `Capture.rate_error` is measured against the **nominal** model, whose error is > `Capture.rate_error` is measured against the **nominal** model, whose error is
> cycle-count dependent — +0.84% at cc=100 and −0.99% at cc=400 on one unit, same > cycle-count dependent, which makes it a configuration diagnostic.
> supply. It is a configuration diagnostic, not a gain measurement, and > `calibrate.py` corrects gain against a measured per-unit reference.
> multiplying a field by `1 + rate_error` is not a correction. `calibrate.py`
> does that against a measured reference instead.
A capture is analysable whether the run ended on `--duration`, on **Ctrl-C**, or
early — it is valid up to wherever it stopped, so the analysis tools can be
pointed at it either way.
## Capture format ## Capture format
A capture stores **only irreducible facts** — the chip's sample count, the host A capture stores **only irreducible facts**, the chip's sample count, the host
clock, and the raw counts — behind a header of configuration. Chip time, elapsed clock and the raw counts, behind a header of configuration. Chip time, elapsed
time, tesla and magnitude are all reconstructed on load by `capture.py`, so there time, tesla and magnitude are all reconstructed on load by `capture.py`, so there
is exactly one source of truth for each. is exactly one source of truth for each.
@ -351,14 +253,13 @@ sample_index,system_time_unix,x_raw,y_raw,z_raw,warning
``` ```
`tesla_per_count` is written with `repr()` so it round-trips through float64 `tesla_per_count` is written with `repr()` so it round-trips through float64
exactly, and is expressed per-count rather than the datasheet's LSB/µT so exactly, and is expressed per-count, so conversion from the datasheet's LSB/µT
conversion is a multiply: `tesla = count * tesla_per_count`. All times are unix is a multiply. All times are unix epoch seconds.
epoch seconds; there is no ISO-8601 anywhere in the data.
The `warning` column carries space-separated flags, empty when the row is fine, The `warning` column carries space-separated flags, empty when the row is fine,
and generalises to future flags. Because it already marks a row as having no and generalises to future flags. Because it already marks a row as having no
data, placeholders carry **zeros rather than blanks**, so `x/y/z` parse as data, placeholders carry **zeros**, which keeps `x/y/z` parsing as integers on
integers on every row without special-casing. every row.
| flag | meaning | | flag | meaning |
|---|---| |---|---|
@ -366,91 +267,16 @@ integers on every row without special-casing.
| `AMBIGUOUS` | the gap ending at this row was of uncertain length, so the index may have slipped from here on | | `AMBIGUOUS` | the gap ending at this row was of uncertain length, so the index may have slipped from here on |
**The two are independent.** A gap measuring 1.35 periods rounds to one, so no **The two are independent.** A gap measuring 1.35 periods rounds to one, so no
placeholder is written — yet it sits far enough from an integer to distrust, and placeholder is written, yet it sits far enough from an integer to distrust, and
that case carries `AMBIGUOUS` on the *real* sample ending the gap, which keeps that case carries `AMBIGUOUS` on the *real* sample ending the gap, which keeps
its data. A placeholder inside an uncertain gap carries both. its data. A placeholder inside an uncertain gap carries both. The flag is a
confidence measure: 1.35 probably is one period and 1.65 probably two, but
The flag is a confidence measure, not a claim that 1.35 and 1.65 are equally deciding which needs neighbouring timestamps and assumptions, so the row carries
likely — 1.35 probably is one period and 1.65 probably two. Deciding which needs the flag and the reader decides.
neighbouring timestamps and assumptions and can still be wrong, which is exactly
why these rows are flagged rather than silently resolved.
`capture.py` interpolates `MISSED` rows on load so the uniform grid the spectra `capture.py` interpolates `MISSED` rows on load so the uniform grid the spectra
depend on survives, and reports how many were substituted so it is never silent. depend on survives, and reports how many were substituted so it is never silent.
## How it works
**The chip samples on its own internal schedule**, unaffected by bus traffic
(§5.8.2 *"this can run in the background"*; §5.7.2 *"This will not affect the
measurement process"*). Two consequences drive the whole design:
- **Read jitter is not sample jitter.** The measurement grid stays uniform even
when host reads are late, so `sample_index` is a grid coordinate.
- **A late read returns the *newer* measurement**, not a delayed one. An
unnoticed miss therefore skips a grid point and silently compresses the time
axis.
So a lost measurement is **recorded, not dropped**: a placeholder row keeps
`sample_index` contiguous and makes the gap explicit. The run continues and
reports the total at the end. Double-counting is structurally impossible —
`HSHAKE` DRC1=1 makes DRDY clear only on a results read, and `configure()`
verifies the readback rather than assuming it.
**The period is calibrated before recording starts.** Counting how many grid
points passed unseen needs the real period, and the TMRC table value is 6–9% out
on this unit — enough to insert the wrong number of placeholders and slip the
index. `calibrate_period()` takes a least-squares slope over one second of
loss-free samples, which pins it to ~0.07%, then keeps refining it from clean
intervals so it follows the oscillator's thermal drift. If no clean stretch can
be found the run aborts before recording anything: that is the honest signal that
the requested rate is not sustainable.
**Counting lost measurements uses two different signals**, because the two
questions have different best answers:
- *Did we lose any?* — the **DRDY bracket**, the span between the last poll
showing DRDY clear and the poll showing it set. This is exact rather than
heuristic: if DRDY reads clear at t꜀ then every earlier measurement has already
been read, measurements complete one period apart, so a bracket narrower than a
period can contain at most one completion — and DRDY going high proves it
contained at least one.
- *How many?* — the **interval since the previous accepted sample**, rounded to
whole periods. Every accepted read sits on a grid point, so that interval is
nearly an exact multiple. The bracket cannot say, since it only reaches back to
the last poll that saw DRDY clear and so discards where the grid is.
The read-to-read interval alone cannot make the first claim: measured here it
reaches 37 ms against a 28.8 ms period — a 29% overshoot from host stalls alone —
while the bracket stays under 10 ms. Thresholding the interval flags healthy
captures as lossy.
**Clock drift is compensated in post-processing, not baked into the file.** The
RC oscillator is *regular* but only accurate to ±7% (§5.2.1). `capture.py`
regresses the host clock on `sample_index` to recover the true period:
```
elapsed_nominal = N * dt_nominal # uniform, but wrongly scaled
elapsed = N * dt_true # uniform and correctly scaled
```
Short-term regularity comes from the chip, long-term rate calibration from the
host. `characterize.py` uses `dt_true` for its frequency axis, since a 6% error
would displace every spectral feature by 6%. Over hours the oscillator drifts
enough that no single slope fits; `capture.py` reports that as `drift_limited`
and says to analyse shorter windows.
**Threading.** The sampling loop does I²C and nothing else, pushing raw counts
onto a bounded queue; a writer thread does unit conversion, CSV formatting,
flushing and the console display. The sampler releases the GIL inside each USB
transfer and must re-acquire it, so Python's default 5 ms switch interval becomes
the jitter floor — `sys.setswitchinterval(0.0005)` cut the worst bracket from
9.96 ms to 5.65 ms. The writer is also reniced out of the way, which needs no
privilege; `--high-priority` additionally raises the sampler, which does.
If the queue ever fills, the run **stops** rather than dropping a row: a missing
row would leave a hole in `sample_index`, and a truncated capture is worth more
than a longer one no tool can load.
## Configuration ## Configuration
**Cycle count is the rate knob, not TMRC.** Two ceilings compete and the slower **Cycle count is the rate knob, not TMRC.** Two ceilings compete and the slower
@ -462,18 +288,15 @@ how *often* one is started.
| cycle count | `3 × (cc/90,000 + 68.7 µs)` per measurement | continuous | | cycle count | `3 × (cc/90,000 + 68.7 µs)` per measurement | continuous |
| TMRC | how often a measurement starts | factor-of-two steps | | TMRC | how often a measurement starts | factor-of-two steps |
> The 90,000 counts/s is the specified figure and holds up; **the 68.7 µs > The 90,000 counts/s is specified and holds up. **The 68.7 µs overhead does
> overhead does not.** Solving the model against two cycle counts on this unit > not**: solving the model on this unit gives 40.6 µs at 3.0 V and 38.1 µs at
> gives 40.6 µs at 3.0 V and 38.1 µs at 3.35 V, so the predicted cc100:cc400 rate > 3.35 V, and it moves with the supply because the same oscillator times it.
> ratio is 1.9% out. The overhead also moves with the supply, because the same
> oscillator times it — see below.
Leaving TMRC faster than the cycle-count ceiling makes the sensor free-run at Leaving TMRC faster than the cycle-count ceiling makes the sensor free-run at
~100% duty and renders TMRC irrelevant. Setting it slower makes the sensor idle, ~100% duty. Setting it slower makes the sensor idle, which costs sensitivity for
which costs sensitivity for nothing: noise after filtering scales as `1/√duty`, nothing: noise after filtering scales as `1/√duty`, and a measured 23% duty cost
and a measured 23% duty cost 1.43× the noise ASD against the same rate reached by 1.43× the noise ASD against the same rate reached by cycle count alone. So the
cycle count alone. So the default in every branch is the fastest TMRC, and default in every branch is the fastest TMRC, and `--rate` derives the cycle count.
`--rate` derives the cycle count from there.
**Cycle count also sets resolution**, which is the real trade: **Cycle count also sets resolution**, which is the real trade:
@ -482,43 +305,15 @@ cycle count alone. So the default in every branch is the fastest TMRC, and
| 50 | 534 Hz | 267 Hz | 50.37 | 0.58 | 89.0% | spectrum, thin dither | | 50 | 534 Hz | 267 Hz | 50.37 | 0.58 | 89.0% | spectrum, thin dither |
| **100** | **283 Hz** | **141 Hz** | **26.17** | **0.79** | **94.2%** | **default** | | **100** | **283 Hz** | **141 Hz** | **26.17** | **0.79** | **94.2%** | **default** |
| 200 | 145 Hz | 73 Hz | 13.35 | 1.10 | 97.0% | resolution, 60 Hz only | | 200 | 145 Hz | 73 Hz | 13.35 | 1.10 | 97.0% | resolution, 60 Hz only |
| 400 | 74 Hz | 37 Hz | 6.74 | 1.54 | 98.5% | mains aliases, fs/4 artefact | | 400 | 74 Hz | 37 Hz | 6.74 | 1.54 | 98.5% | mains aliases, fs/4 artifact |
*dither* is the sensor's own noise in LSB. Below roughly 0.2 LSB the quantiser *dither* is the sensor's own noise in LSB; below roughly 0.2 LSB the quantiser
stops being dithered and averaging no longer recovers sub-LSB resolution. stops being dithered and averaging no longer recovers sub-LSB resolution.
`characterize.py` prints the measured figure for any capture, and cycle count
200 is the fallback where its dither check shows the LSB column flattening out. The notebook has the reasoning behind the default and behind preferring
to decimate a fast capture over sampling slowly.
**Measured, not assumed:** at cc=100 the dither came out at 0.65–0.81 LSB and **Bus speed is independent of the rate**, appearing in neither mechanism. What
averaging 1,024 samples reached **0.035–0.050 LSB** — a factor of 20 below the
step, within 1.5–2.1× of the ideal `1/√n`. The quantiser is not stalling.
`characterize.py` prints this for any capture; see
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md) §2.
**The default is cycle count 100 at 750 kHz**, which runs the sensor at its own
~283 Hz ceiling. It sits deliberately between the two things pulling in opposite
directions:
- **Against cc=200** it costs 1.5% in post-filter noise for 1.9× the spectrum
(141 Hz of Nyquist against 73 Hz). Worth taking, because **aliased interference
cannot be filtered out afterwards at any cycle count** — at 283 Hz both mains
and its second harmonic are in band and can be notched.
- **Against cc=50** it gives up half the spectrum and buys 36% more dither
margin. cc=50 is right when something above 141 Hz needs identifying; it is not
the right default, because its dither margin is the thinnest here and the only
one still unmeasured.
Fall back to cycle count 200 if `characterize.py`'s dither check shows the LSB
column flattening rather than continuing to fall.
**Prefer decimating a fast capture over sampling slowly.** Measured on a cc=100
capture decimated by 4 against a natively-recorded cc=400 one: decimation changes
a capture's own broadband floor by **+0.3 to +0.5%**, so the two are equivalent
for noise — and decimation is strictly better on everything else. The natively
slow capture folds 60 Hz irrecoverably to 13–17 Hz and carries sample-locked
lines at fs/4 and fs/2 that the decimated path does not have at all. `compare.py`
prints this comparison whenever two captures differ by an integer cycle-count
factor.
**Bus speed is independent of the rate** — it appears in neither mechanism. What
it sets is latency: how long a read takes, hence how tightly DRDY can be it sets is latency: how long a read takes, hence how tightly DRDY can be
timestamped and how much margin there is against a stall. Host cost is timestamped and how much margin there is against a stall. Host cost is
`bus time + ~0.6 ms` of fixed USB round trip, so faster is simply better. `bus time + ~0.6 ms` of fixed USB round trip, so faster is simply better.
@ -536,14 +331,14 @@ bounds are enforced by `rm3100.plan()`.
### Configuration warnings ### Configuration warnings
`rm3100.plan()` resolves a configuration, `logger.py` prints its derivation, then `rm3100.plan()` resolves a configuration, `logger.py` prints its derivation, then
checks it against six known traps. Each is a *silent* failure — the capture checks it against six known traps. Each is a *silent* failure: the capture
completes, the numbers look plausible, and the defect only shows up afterwards. completes, the numbers look plausible, and the defect shows up afterwards.
So each is reported on stderr and **nothing is fixed automatically**: changing a So each is reported on stderr and **nothing is fixed automatically**: changing a
setting that was asked for would hide the problem behind a configuration change. setting that was asked for would hide the problem behind a configuration change.
| Warning | Trigger | Why it matters | | Warning | Trigger | Why it matters |
|---|---|---| |---|---|---|
| cycle count below the recommended 50 | `cc < RECOMMENDED_MIN_CYCLE_COUNT` | dither thins toward the ~0.2 LSB where averaging stops recovering sub-LSB resolution | | cycle count below the recommended 50 | `cc < RECOMMENDED_MIN_CYCLE_COUNT` | duty falls, and post-filter noise goes as `1/√duty` |
| TMRC governs and the sensor idles | TMRC-governed, idle > 20% | idle time buys nothing; measured 1.43× the ASD at 23% duty | | TMRC governs and the sensor idles | TMRC-governed, idle > 20% | idle time buys nothing; measured 1.43× the ASD at 23% duty |
| rate differs from the one requested | \|error\| > 2% | the run silently uses the ceiling, and every derived figure moves with it | | rate differs from the one requested | \|error\| > 2% | the run silently uses the ceiling, and every derived figure moves with it |
| Nyquist below 60 Hz | rate < 120 Hz | mains folds onto signal and no later filter undoes it | | Nyquist below 60 Hz | rate < 120 Hz | mains folds onto signal and no later filter undoes it |
@ -557,29 +352,29 @@ the line still visible.
## Diagnostics ## Diagnostics
`diagnose-comms.py` walks the chain — USB → bus → identity → registers → BIST → `diagnose-comms.py` walks the chain, USB → bus → identity → registers → BIST →
live read — and stops at the first failure, so a fault points at a specific wire. live read, and stops at the first failure, so a fault points at a specific wire.
Otherwise, symptoms map onto causes: Otherwise, symptoms map onto causes:
| Symptom | Cause | | Symptom | Cause |
|---|---| |---|---|
| `Cannot claim CH347 interface 2: Access denied` | udev rule missing or not yet applied to this node — replug the adapter, then run `./setup.sh`, which reports whether `uaccess` or group membership granted access and diagnoses whichever fell through. On Bazzite also try `sudo udevadm control --reload-rules && sudo udevadm trigger`. Do **not** reach for `sudo`: it works as a normal user, and masking a permissions problem with root only defers it | | `Cannot claim CH347 interface 2: Access denied` | udev rule missing or not yet applied to this node; replug the adapter, then run `./setup.sh`. On Bazzite also try `sudo udevadm control --reload-rules && sudo udevadm trigger`. Do **not** reach for `sudo`: it works as a normal user, and masking a permissions problem with root only defers it |
| `No CH347 adapter found` | not plugged in, or not in Mode 1 | | `No CH347 adapter found` | not plugged in, or not in Mode 1 |
| `Could not find N s of loss-free samples` | the host cannot sustain this rate — lower it (`--rate`, or a higher `--tmrc`) or raise `--bus-speed` | | `Could not find N s of loss-free samples` | the host cannot sustain this rate; lower it (`--rate`, or a higher `--tmrc`) or raise `--bus-speed` |
| `N measurement(s) were lost` | host stalls during the run. Same fixes; the capture is still usable, with explicit gaps | | `N measurement(s) were lost` | host stalls during the run. Same fixes; the capture is still usable, with explicit gaps |
| `N gap(s) could not be counted confidently` | the index may have slipped — re-record before doing spectral work | | `N gap(s) could not be counted confidently` | the index may have slipped; re-record before doing spectral work |
| `no capture header found` | a capture predating the header format — re-record it | | `no capture header found` | a capture predating the header format; re-record it |
| `sample_index is not contiguous` | the file was truncated mid-row, damaged, or hand-edited | | `sample_index is not contiguous` | the file was truncated mid-row, damaged, or hand-edited |
| `HSHAKE did not take` | the I²C write is unreliable; try a lower `--bus-speed` | | `HSHAKE did not take` | the I²C write is unreliable; try a lower `--bus-speed` |
| Adapter opens, `--scan-only` finds nothing | **I2CEN not tied high** (most likely), or SDA/SCL swapped, or no bus pull-ups | | Adapter opens, `--scan-only` finds nothing | **I2CEN not tied high** (most likely), or SDA/SCL swapped, or no bus pull-ups |
| Found at 0x23, REVID `0x22`, registers fine, but DRDY never sets and results are all zero | **AVDD/VDD not powered** — confirm with BIST | | Found at 0x23, REVID `0x22`, registers fine, but DRDY never sets and results are all zero | **AVDD/VDD not powered**; confirm with BIST |
| Cycle-count read-back mismatch | bus integrity — try a lower speed | | Cycle-count read-back mismatch | bus integrity; try a lower speed |
**BIST is the definitive test for the analog side** (§5.6.1). Write `0x8F` to **BIST is the definitive test for the analog side** (§5.6.1). Write `0x8F` to
BIST (STE=1, max timeout and periods), write `0x70` to POLL, wait, then read BIST BIST (STE=1, max timeout and periods), write `0x70` to POLL, wait, then read BIST
back: bits 4/5/6 are XOK/YOK/ZOK, and `1` means that axis's LR oscillator ran. back: bits 4/5/6 are XOK/YOK/ZOK, and `1` means that axis's LR oscillator ran.
All zeros means the coils are not oscillating, pointing at AVDD or the REXT All zeros means the coils are not oscillating, which points at AVDD or the REXT
timing resistor rather than anything on the I²C side. timing resistor.
A healthy total field magnitude is roughly **25–65 µT** (Earth's field). Near A healthy total field magnitude is roughly **25–65 µT** (Earth's field). Near
zero, railed, or wildly out of range means the decode or the gain is wrong rather zero, railed, or wildly out of range means the decode or the gain is wrong rather
@ -590,84 +385,28 @@ than merely "data arrived".
| File | Role | | File | Role |
|---|---| |---|---|
| `setup.sh` | udev rule, venv, dependencies, node permission check | | `setup.sh` | udev rule, venv, dependencies, node permission check |
| `ch347.py` | CH347 I²C transport — USB only, knows nothing about the sensor | | `ch347.py` | CH347 I²C transport; USB only, knows nothing about the sensor |
| `rm3100.py` | RM3100 driver and configuration model — knows nothing about USB | | `rm3100.py` | RM3100 driver and configuration model; knows nothing about USB |
| `logger.py` | CLI: scan → identify → configure → calibrate → log, recording lost measurements as explicit placeholders | | `logger.py` | CLI: scan → identify → configure → calibrate → log, recording lost measurements as explicit placeholders |
| `capture.py` | the only capture reader — parses the header, rebuilds tesla and both time bases | | `capture.py` | the only capture reader; parses the header, rebuilds tesla and the time base |
| `plot.py` | four-panel plot of a capture: X, Y, Z and the norm | | `plot.py` | four-panel plot of a capture: X, Y, Z and the norm |
| `characterize.py` | noise floor: spectral density, Allan deviation, residuals, read latency | | `characterize.py` | noise floor and stability: spectral density, Allan deviation, residuals, read latency, oscillator drift |
| `sweep.py` | sweep target rates, reporting measured rate, LSB, noise, duty and bus use per point | | `sweep.py` | sweep target rates, reporting measured rate, LSB, noise, duty and bus use per point |
| `compare.py` | A/B captures — fractional noise, plus the checks that separate a gain change from a moved sensor | | `compare.py` | A/B captures: fractional noise, plus the checks that separate a gain change from a moved sensor |
| `calibrate.py` | convert a raw capture to nanotesla, applying a per-unit calibration if one is given; also holds the calibration model | | `calibrate.py` | convert a raw capture to nanotesla, applying a per-unit calibration if one is given; also holds the calibration model |
| `diagnose-comms.py` | walks USB → bus → identity → registers → BIST → live read, stopping at the first failure | | `diagnose-comms.py` | walks USB → bus → identity → registers → BIST → live read, stopping at the first failure |
| `tests/` | pytest suite; needs no hardware | | `tests/` | pytest suite; needs no hardware |
| `Noise_Floor_Testing/` | captures, figures and [NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md) — the supply and rate-scaling analysis. Untracked: `.gitignore` excludes it | | `Noise_Floor_Testing/` | the lab notebook, captures, figures and the supply analysis. Only `NOTES.md` is tracked |
The adapter/sensor split is deliberate: `ch347.py` is a general I²C master usable The adapter/sensor split is deliberate: `ch347.py` is a general I²C master usable
with any device, and `rm3100.py` needs only a bus object exposing with any device, and `rm3100.py` needs only a bus object exposing
`write(addr, data)` and `read(addr, count)` — optionally `write_read()`, which it `write(addr, data)` and `read(addr, count)`, optionally `write_read()`, which it
prefers when available. prefers when available.
## Implementation notes The CH347 wire format, the RM3100 register quirks and the reasoning behind the
time base are in
Things that cost time to work out, recorded so they don't have to be again. [NOTES.md](Noise_Floor_Testing/NOTES.md#design-rationale-moved-out-of-the-readme);
`tests/test_ch347.py` pins every framing form byte for byte.
### CH347 access
The adapter presents three USB interfaces. Interfaces 0 and 1 are CDC-ACM and the
kernel binds them as `/dev/ttyACM0` (the UART). **Interface 2** is vendor-class,
carries I2C/SPI/JTAG with no kernel driver bound, and libusb can claim it
directly with nothing to detach. `ch347.py` deliberately never calls
`set_configuration()` — the device is already configured, and re-setting it would
disturb the CDC-ACM interfaces driving the UART.
There is no usable off-the-shelf option: no CH347 kernel driver exists (the
in-tree `spi-ch341` is for `1a86:5512`, a different chip), the PyPI `ch347`
package wraps a Windows DLL, and `ch347api` supports only HID mode (`55dc`), not
this vendor-bulk Mode 1. The framing follows the `aystarik/ch347-i2c-spi-gpio`
Linux driver, whose id-table entry
`USB_DEVICE_INTERFACE_NUMBER(0x1a86, 0x55db, 0x02)` matches this device exactly.
Wire format — bulk OUT `0x06`, bulk IN `0x86`, max 63 bytes per transfer:
| Purpose | Bytes out | Bytes back |
|---|---|---|
| Set speed | `AA 6<speed> 00` | 0 |
| Write n | `AA 74 (80\|n+1) (addr<<1) <data> 75 00` | n+1 |
| Read n | `AA 74 81 ((addr<<1)\|1) [C0\|(n-1)] C0 75 00` | n+1 |
| Write n then read m | `AA 74 (80\|n+1) (addr<<1) <data> 74 81 ((addr<<1)\|1) [C0\|(m-1)] C0 75 00` | n+2+m |
| Probe | `AA 74 81 (addr<<1) 75 00` | 1 |
**Every returned byte must be `1`** — that is the per-byte ACK. On a read, the
leading bytes are address ACKs and the rest is payload. `tests/test_ch347.py`
pins all five forms byte for byte.
### RM3100 quirks
- **Register reads use the plain address, not `|0x80`.** §5 describes the SPI
convention of adding 0x80, but the I2C diagram in §5.8.4 writes `0x24`
literally. Only 7 bits are decoded, so both work; the plain form is used here.
- **A register read uses a repeated START**, not the STOP-then-START the manual
draws (§4.5.2, §5.8.4). The part accepts it — verified against REVID and the
measurement registers — and it halves the USB round trips, which is what sets
the sample-rate ceiling. `rm3100.read_reg()` falls back to the manual's form on
a bus that cannot do it.
- **CMM = 0x79** for all three axes. Table 5-1 describes bit 3 as reserved-zero,
but the manual's own examples (§5.7.2, §5.8.3) set it. The examples win.
- **HSHAKE is set to 0x0A** (`DRC0=0`, `DRC1=1`) during init. The 0x1B default has
`DRC0=1`, meaning *any* register write clears DRDY — including the pointer write
that reading STATUS itself requires, so polling could never observe DRDY set.
With `DRC0=0`, DRDY clears only on a results read, which is what makes
exactly-once sampling possible.
- **Gain** is `0.3671 × cycle_count + 1.5` LSB/µT, a linear fit to Table 3-1
(50→20, 100→38, 200→75; reproduces all three within a count).
- **The rate model divisor is specified, not fitted.** Table 3-1 gives a 180 kHz
circuit oscillation and §4.1 measures each cycle count in *both* bias
directions, so one count costs two oscillations — hence 90,000 counts/s. The
68.7 µs per-axis overhead has no specified value and was measured; it is why
the naive `rate × cc` "constant" drifts from 84,429 at cc=100 to 89,191 at 1200.
- Measurements are three **24-bit big-endian two's-complement** values, read as
9 bytes from 0x24 using the sensor's register auto-increment.
## Tests ## Tests
@ -686,38 +425,11 @@ failures are invisible downstream: a stall of 1.1 periods must lose nothing, 2.1
must record one placeholder, and 1.35 must flag the sample as `AMBIGUOUS` without must record one placeholder, and 1.35 must flag the sample as `AMBIGUOUS` without
inserting anything. inserting anything.
## Status ---
- [x] udev rule, venv and dependency setup
- [x] CH347 vendor protocol — verified byte-for-byte against the kernel driver
- [x] I²C bus scan, sensor identified at 0x23, REVID `0x22`
- [x] Analog section verified — BIST reports `XOK=YOK=ZOK=1`
- [x] Exactly-once sampling, chip-grid time base, explicit gaps for lost samples
- [x] Verified on hardware: exact grid, −6.11% drift recovered, 0.017% run-to-run
- [x] Calibrated period, configuration warnings, cycle-count rate model
- [x] 3.0 V LDO built and compared — noise indistinguishable once normalised
- [x] Test suite covering everything that does not need the adapter
- [x] Interleaved supply A/B at two cycle counts — **scale factor quantified at
−7.1% for +0.347 V**, rate at +4.94%
- [x] Dither margin at cc=100 checked against real data — averaging reaches
0.035 LSB, so the quantiser is not stalling
- [x] Filter-and-decimate shown equivalent to sampling slowly (+0.5% on the
broadband floor), and better on aliasing and artefacts
- [ ] Re-test the LDO with the sensor **clamped**, to separate gain from movement
— every pair in the A/B still shows 2.8–23° of rotation
- [ ] Identify the **fs/4 artefact** that appears at cc=400 and not at cc=100;
a cc=200 run at both supplies would say whether it scales with cycle count
- [x] Per-unit timing and gain calibration — `calibrate.py`, written by
`sweep.py --calibration-out`, nothing corrected by default
- [ ] Measure a calibration on the bench and check the oscillator against a
third rail voltage, which would separate V from V² in the gain scaling
- [ ] Deal with 60 Hz coupling at source
- [ ] Reliability testing over extended runs on bare metal
Captures written before the current header format are unreadable and must be Captures written before the current header format are unreadable and must be
re-recorded — `capture.py` says so explicitly rather than guessing. Figures re-recorded; `capture.py` says so explicitly. Figures quoted in the notebook
quoted in [NOTES.md](NOTES.md) came from those files and stand as historical came from those files and stand as historical measurements only.
measurements only.
A proper Python API is planned; these modules are the initial-communication A proper Python API is planned; these modules are the initial-communication
milestone, deliberately kept simple. milestone, deliberately kept simple.

309
calibrate.py
View file

@ -7,51 +7,30 @@
Without `--calibration` this only changes the format: counts become nanotesla Without `--calibration` this only changes the format: counts become nanotesla
using the `tesla_per_count` the capture already carries, and nothing is using the `tesla_per_count` the capture already carries, and nothing is
corrected. That is the default deliberately -- see "What is not corrected". corrected. That is deliberate: the corrections are real but rest on a small
number of captures taken with a sensor that moved between runs, so they are
applied only when asked for, and every factor carries the standard deviation it
earned. See Noise_Floor_Testing/NOTES.md for the measurements behind them.
**Why a calibration is per-unit.** The chip times everything from one oscillator, **A calibration is per unit and per supply.** The chip times everything from one
and `rm3100`'s model of it, `period = 3 x (cycle_count / C + overhead)`, uses a oscillator, and `rm3100`'s model of it, `period = 3 x (cycle_count / C +
specified C and an overhead measured on one rig. Neither fits every part. Solving overhead)`, uses a specified C and a nominal overhead that fit no particular
the model against one unit's two cycle counts gave C = 88,546 with a 40.6 us part. A calibration records a measured pair plus a reference oscillator
overhead at 3.006 V, against the model's 90,000 and 68.7 us -- and C moved to frequency; the correction is how far this capture's oscillator sits from that
92,889 when the supply moved to 3.353 V, because the oscillator does. reference. Gain moves with the oscillator, so a capture taken at another supply
reads a different field for the same magnet.
That matters beyond timing. Gain moves with the oscillator, so a capture taken at
a different supply reads a different field for the same magnet. A calibration
records the timing pair and a reference oscillator frequency, and the correction
is how far this capture's oscillator sits from that reference.
Once the timing pair is right, **one capture is enough** to recover the Once the timing pair is right, **one capture is enough** to recover the
oscillator: rearranging the model gives `C = cycle_count / (period/3 - overhead)`, oscillator: `C = (cycle_count + n) / (period/3)`, where n is the overhead in
and a 1 us error in the overhead moves C by only 0.09% at cycle count 100. With counts. `gain_exponent` is 1 because the measurement interval is clocked
the model's 28 us error it moves by 2.5%, and by a *different* amount at each entirely by that oscillator. The overhead is a fixed *count* of cycles, so the
cycle count, which is what makes the uncalibrated figures unusable for this. same number of counts is collected over a window whose length goes as 1/f, and
gain follows integration time. `oscillator_hz` rests on the same premise, which
keeps the two consistent.
**Why the exponent is 1.** The measurement interval is clocked entirely by that The output is a different format from a raw capture and `capture.py` refuses to
oscillator: the per-axis overhead measures 3.60 counts at 3.006 V and 3.54 at read it, which keeps raw captures the one source of truth. The header carries
3.353 V while its *duration* moves 6%, so the same number of counts is always the source's SHA-256, so a derived file can be tied back to what produced it.
collected over a window whose length goes as 1/f. Gain then follows integration
time, giving `gain ~ (cc + n) / f` -- exponent exactly 1. Measured rates agree:
the supply rate ratio is the same at both cycle counts to 0.04%, which a
fixed-time overhead would not produce. It is favoured rather than proven, at 0.3
sigma against 1.2 for a fixed-time overhead; thermal drift is what limits it.
**Why there is a gain offset.** Table 3-1's `0.3671*cc + 1.5` LSB/uT implies the
gain goes as `cc + 4.086` in count units. Solving the same-supply cycle-count
contrast on one unit gives 0.90 and -0.09 -- near zero, so the overhead counts
cost time without integrating field. Leaving it at the datasheet value accounts
for only half of what the correction can remove.
**What is not corrected, and why nothing is by default.** All of it was measured
on four captures, two per supply, with a sensor that rotated between runs by 2.8
to 23 degrees, and every capture drifts 186 to 1241 ppm through its own thermal
warm-up. So it is applied only when a calibration is named, every factor arrives
with the standard deviation it earned, and the output records what was done.
The output is a different format from a raw capture and `capture.py` will refuse
to read it, which is intended: raw captures stay the one source of truth, and a
derived file that drifts from its source should be detectable rather than
plausible. The header carries the source's SHA-256 for that reason.
""" """
import argparse import argparse
@ -70,7 +49,7 @@ import characterize as ch
import rm3100 import rm3100
# Bumped when either file format changes incompatibly, so a reader can refuse # Bumped when either file format changes incompatibly, so a reader can refuse
# rather than misinterpret. # a file it would misread.
CALIBRATION_VERSION = 2 CALIBRATION_VERSION = 2
CALIBRATED_VERSION = 1 CALIBRATED_VERSION = 1
@ -78,11 +57,6 @@ CALIBRATED_VERSION = 1
# no reason to hold one in memory twice. # no reason to hold one in memory twice.
HASH_BLOCK = 1 << 20 HASH_BLOCK = 1 << 20
# Sub-windows used to measure how much the oscillator moved during a capture.
# Eight over a five-minute run is ~40 s each, long enough to fit a rate against
# and short enough to see a thermal transient.
STABILITY_WINDOWS = 8
class CalibrationError(Exception): class CalibrationError(Exception):
pass pass
@ -91,10 +65,10 @@ class CalibrationError(Exception):
class Estimate(namedtuple("Estimate", "value sd")): class Estimate(namedtuple("Estimate", "value sd")):
"""A number and its standard deviation, carried together. """A number and its standard deviation, carried together.
Every corrected field here rests on a chain of measurements, and a residual Every corrected field rests on a chain of measurements, and a residual only
only means something against the uncertainty of the thing that was supposed means something against the uncertainty of the thing that was supposed to
to remove it. Returning the two together is what makes that comparison remove it. Carrying the two together puts that comparison at the point of
possible at the point of use rather than in a comment. use.
""" """
__slots__ = () __slots__ = ()
@ -103,9 +77,6 @@ class Estimate(namedtuple("Estimate", "value sd")):
def relative(self): def relative(self):
return self.sd / abs(self.value) if self.value else float("inf") return self.sd / abs(self.value) if self.value else float("inf")
def __format__(self, spec):
return f"{self.value:{spec or '.6g'}} +/- {self.sd:{spec or '.2g'}}"
Calibration = namedtuple("Calibration", [ Calibration = namedtuple("Calibration", [
"counts_per_second", # C in the timing model, this unit's own "counts_per_second", # C in the timing model, this unit's own
@ -113,7 +84,7 @@ Calibration = namedtuple("Calibration", [
"reference_oscillator_hz", # C at which the gain is taken as correct "reference_oscillator_hz", # C at which the gain is taken as correct
"reference_cycle_count", # cycle count at which it is taken as correct "reference_cycle_count", # cycle count at which it is taken as correct
"gain_exponent", # gain ~ (cc + offset) / f ** exponent "gain_exponent", # gain ~ (cc + offset) / f ** exponent
"gain_offset_counts", # the "+ n" -- see gain_factor() "gain_offset_counts", # the "+ n"; see gain_factor()
"counts_per_second_sd", "counts_per_second_sd",
"axis_overhead_s_sd", "axis_overhead_s_sd",
"gain_exponent_sd", "gain_exponent_sd",
@ -133,14 +104,13 @@ def calibration(counts_per_second, axis_overhead_s,
A calibration whose reference is its own oscillator frequency corrects A calibration whose reference is its own oscillator frequency corrects
nothing on the unit and supply it was taken at, and corrects other captures nothing on the unit and supply it was taken at, and corrects other captures
relative to it. That is the useful default: the reference is a choice of relative to it. That is the useful default: without an absolute field
where "right" is, and the only defensible choice without an absolute field standard, the only defensible choice of where "right" is, is the condition
standard is the condition the gain was characterised at. the gain was characterised at.
The uncertainties all default to zero, so a calibration that makes no claim The uncertainties default to zero, so a calibration making no claim about
about its own accuracy still works -- and every field it corrects then its own accuracy still works. Every field it corrects then reports a
reports a standard deviation of zero, which is a visible claim rather than a standard deviation of zero, which states that claim on the face of the file.
silent one.
""" """
return Calibration( return Calibration(
counts_per_second=float(counts_per_second), counts_per_second=float(counts_per_second),
@ -163,8 +133,7 @@ def _finite(value, field):
"""A field that must parse as a real number. Booleans are not numbers here. """A field that must parse as a real number. Booleans are not numbers here.
JSON's `true` reaches Python as a bool, which floats to 1.0 without JSON's `true` reaches Python as a bool, which floats to 1.0 without
complaint -- a silent way for a malformed calibration to become a plausible complaint, so booleans are rejected here.
one.
""" """
if isinstance(value, bool) or not isinstance(value, (int, float, str)): if isinstance(value, bool) or not isinstance(value, (int, float, str)):
raise CalibrationError(f"{field}: {value!r} is not a number") raise CalibrationError(f"{field}: {value!r} is not a number")
@ -207,7 +176,7 @@ def load_calibration(path):
f"{path}: rm3100_calibration is {version!r}, this reads " f"{path}: rm3100_calibration is {version!r}, this reads "
f"{CALIBRATION_VERSION}. Version 1 carried a per-cycle-count gain " f"{CALIBRATION_VERSION}. Version 1 carried a per-cycle-count gain "
"table; version 2 replaces it with gain_offset_counts, which is " "table; version 2 replaces it with gain_offset_counts, which is "
"not derivable from it -- re-measure rather than convert.") "not derivable from it; re-measure it.")
missing = {"counts_per_second", "axis_overhead_s"} - raw.keys() missing = {"counts_per_second", "axis_overhead_s"} - raw.keys()
if missing: if missing:
raise CalibrationError(f"{path}: missing {', '.join(sorted(missing))}") raise CalibrationError(f"{path}: missing {', '.join(sorted(missing))}")
@ -267,89 +236,6 @@ def save_calibration(path, cal):
return path return path
def window_rates(cap, seconds):
"""Sample rate fitted independently on each consecutive window of a capture.
Returns (centres, rates) -- seconds from the start of the capture, and Hz --
with one point per window that held enough samples to fit.
`Capture.dt_true` is a single straight line through a run whose rate is
moving. This is the same fit repeated on pieces short enough that the
movement within any one of them is small, which is what turns the drift from
an error on the whole-capture number into something measurable in its own
right.
"""
seconds = float(seconds)
start = float(cap.elapsed[0])
if seconds <= 0 or cap.duration < 2 * seconds:
return np.array([]), np.array([])
centres, rates = [], []
for w in range(int(cap.duration // seconds)):
low = start + w * seconds
try:
rate = cap.restrict(low, low + seconds).true_rate_hz
except capture.CaptureError:
continue
rates.append(rate)
centres.append((w + 0.5) * seconds)
return np.array(centres), np.array(rates)
def rate_stability(cap, windows=STABILITY_WINDOWS):
"""Fractional spread of the sample rate across sub-windows of a capture.
The chip's RC oscillator warms up, and every capture here drifts downward
through its own run -- 186 ppm on the quietest, 1241 ppm on the worst. A
single fitted `dt_true` is the average over that transient, so this is the
honest error bar on it, and the input to every uncertainty downstream.
It is not the same thing as `Capture.drift_limited`, which asks whether one
straight line describes the capture at all. This asks how far the rate moved
while it did. Nor is it `rate_allan`, which asks how much of that movement
is real: a spread this wide is only meaningful against the noise of the
measurement that produced it.
"""
if len(cap.sample_index) < 64 * windows:
return 0.0
_, rates = window_rates(cap, cap.duration / windows)
if len(rates) < 2:
return 0.0
return float((rates.max() - rates.min()) / rates.mean())
def rate_allan(cap, taus):
"""Two-sample (Allan) deviation of the measured sample rate, fractional.
For each averaging time tau the rate is fitted on consecutive tau-second
windows and the deviation taken between neighbours:
sigma_y(tau) = sqrt( mean( (y[k+1] - y[k])^2 ) / 2 )
Differences between neighbours rather than a spread about the mean, for the
same reason `characterize.white_sd` differences rather than taking a
standard deviation: on a series that drifts, the spread measures the drift.
Here both effects are wanted and the shape of the curve separates them.
Falling with tau is measurement noise averaging down; rising with tau is
drift the measurement has begun to resolve, and a rise proportional to tau
specifically is a deterministic frequency ramp rather than a random walk.
The minimum is therefore the best the rate can be known, and the tau where
it falls is how long it is worth measuring for.
Returns (taus, sigma_y) over the taus that yielded three or more windows.
Two windows give a single difference, which is not an estimate of anything.
"""
kept, devs = [], []
for tau in taus:
_, rates = window_rates(cap, tau)
if len(rates) < 3:
continue
y = rates / rates.mean()
kept.append(float(tau))
devs.append(float(np.sqrt(np.mean(np.diff(y) ** 2) / 2)))
return np.array(kept), np.array(devs)
TimingFit = namedtuple("TimingFit", "counts_per_second axis_overhead_s " TimingFit = namedtuple("TimingFit", "counts_per_second axis_overhead_s "
"counts_per_second_sd axis_overhead_s_sd " "counts_per_second_sd axis_overhead_s_sd "
"residual_ppm points") "residual_ppm points")
@ -359,17 +245,16 @@ def fit_timing(cycle_counts, periods, period_sd=None):
"""Solve `period = AXES * (cycle_count / C + overhead)` for both terms. """Solve `period = AXES * (cycle_count / C + overhead)` for both terms.
`periods` are whole sample periods in seconds, not per-axis. `period_sd` is `periods` are whole sample periods in seconds, not per-axis. `period_sd` is
the fractional uncertainty on each -- `rate_stability()` is where it comes the fractional uncertainty on each, which `rate_stability()` supplies.
from.
Both terms are needed. Holding C at the specified 90,000 and solving for the Both terms are solved for, since the misfit sits in both: holding C at the
overhead alone gave 58.9 us at cycle count 100 and 113.6 us at 400 on the specified 90,000 gives 58.9 us of overhead at cycle count 100 and 113.6 us
same unit -- no single value fits, because the misfit is in both. at 400 on the same unit.
**Two points determine both terms exactly and leave no residual**, so a bad **Two points determine both terms exactly and leave no residual**, so
point cannot be detected: `residual_ppm` is 0 and means nothing. That case `residual_ppm` is 0 and carries no information there. That case requires
requires `period_sd`, so the uncertainty is at least stated rather than `period_sd`, which states the uncertainty outright. Three or more points
fabricated from a perfect fit. Three or more points carry their own. carry their own.
""" """
counts = np.asarray(cycle_counts, dtype=float) counts = np.asarray(cycle_counts, dtype=float)
per_axis = np.asarray(periods, dtype=float) / rm3100.AXES per_axis = np.asarray(periods, dtype=float) / rm3100.AXES
@ -418,33 +303,58 @@ def fit_timing(cycle_counts, periods, period_sd=None):
float(np.abs(residual / per_axis).max() * 1e6), len(counts)) float(np.abs(residual / per_axis).max() * 1e6), len(counts))
def overhead_counts(cal):
"""The calibration's per-axis overhead expressed in oscillator cycles.
The overhead is stored as a duration, which is what `fit_timing` solves for,
and that duration belongs to the C it was measured with: the overhead is a
fixed *count* of cycles whose duration goes as 1/f. See `oscillator_hz`.
"""
return cal.axis_overhead_s * cal.counts_per_second
def oscillator_hz(cap, cal, period_sd=None): def oscillator_hz(cap, cal, period_sd=None):
"""This capture's oscillator frequency, with its uncertainty. """This capture's oscillator frequency, with its uncertainty.
Inverts `period = AXES * (cycle_count / C + overhead)`. The overhead comes Inverts the timing model. The overhead comes from the calibration, a single
from the calibration because it cannot be recovered from a single cycle cycle count leaving one equation in two unknowns, and it is the term the
count -- one equation, two unknowns -- and it is the term the nominal model nominal model gets most wrong.
gets most wrong.
**The overhead is carried as a count of cycles**, the same premise the gain
exponent rests on, so the model keeps it inside the count and closes in one
step:
period / AXES = (cycle_count + overhead_counts) / C
Written with the calibration's *duration* instead, the two forms disagree by
0.23% on a figure whose propagated standard deviation is 0.30%.
`period_sd` is the fractional uncertainty on the measured period; it `period_sd` is the fractional uncertainty on the measured period; it
defaults to measuring it from the capture with `rate_stability()`. defaults to measuring it from the capture with `rate_stability()`.
""" """
per_axis = cap.dt_true / rm3100.AXES - cal.axis_overhead_s overhead = overhead_counts(cal)
if per_axis <= 0: # The overhead is a handful of counts against a cycle count of 30 or more.
# One that rivals the cycle count would set the recovered frequency, so the
# calibration is refused.
if overhead >= cap.cycle_count:
raise CalibrationError( raise CalibrationError(
f"{cap.path}: a {cap.dt_true * 1e3:.3f} ms period leaves no time " f"{cap.path}: the calibration's {cal.axis_overhead_s * 1e6:.1f} us "
f"for {cap.cycle_count} counts once the calibration's " f"overhead is {overhead:,.1f} counts against a "
f"{cal.axis_overhead_s * 1e6:.1f} us overhead is removed. The " f"{cap.cycle_count}-count measurement, so the overhead, not the "
"calibration does not describe this capture.") "measurement, would set the answer. The calibration does not "
"describe this capture.")
counts = cap.cycle_count + overhead
per_axis = cap.dt_true / rm3100.AXES
if period_sd is None: if period_sd is None:
period_sd = rate_stability(cap) period_sd = ch.rate_stability(cap)
value = cap.cycle_count / per_axis value = counts / per_axis
# Two independent contributions: how well the period was measured, and how # Two independent contributions: how well the period was measured, and how
# well the overhead subtracted from it is known. The second dominates at low # well the overhead added to the count is known. The second dominates at low
# cycle counts, where the overhead is a larger share of the per-axis time. # cycle counts, where the overhead is a larger share of the total count.
from_period = cap.dt_true * period_sd / rm3100.AXES # Both go into fractional terms first, since they enter from opposite ends.
relative = math.hypot(from_period, cal.axis_overhead_s_sd) / per_axis from_period = period_sd
return Estimate(value, value * relative) from_overhead = cal.axis_overhead_s_sd * cal.counts_per_second / counts
return Estimate(value, value * math.hypot(from_period, from_overhead))
def gain_factor(cap, cal, period_sd=None): def gain_factor(cap, cal, period_sd=None):
@ -454,18 +364,20 @@ def gain_factor(cap, cal, period_sd=None):
oscillator how far this capture's clock sits from the reference, raised oscillator how far this capture's clock sits from the reference, raised
to the fitted exponent. The measurement interval is clocked to the fitted exponent. The measurement interval is clocked
entirely by that oscillator -- the per-axis overhead measures entirely by that oscillator, which predicts an exponent of
3.57 counts at both supplies, constant, while its *duration* exactly 1.
moves 6% -- so the same number of counts is always collected
over a window whose length goes as 1/f. That predicts an
exponent of exactly 1.
gain shape Table 3-1's `0.3671*cc + 1.5` LSB/uT implies the gain goes as gain shape Table 3-1's `0.3671*cc + 1.5` LSB/uT implies the gain goes as
`cc + 4.086` in count units. Measured on one unit it is 0 to `cc + 4.086` in count units, where measurement puts it near
0.9, so the overhead counts cost time without integrating zero, the overhead counts costing time while integrating no
field. `gain_offset_counts` is that "+ n". field. `gain_offset_counts` is that "+ n".
reference the shape term is only known up to an overall scale, since reference the shape term is only known up to an overall scale, since
`gain = A(cc + n)` leaves A free. `reference_cycle_count` `gain = A(cc + n)` leaves A free. `reference_cycle_count`
pins it: the correction is exactly 1 there. pins it: the correction is exactly 1 there.
**A matched-cycle-count contrast cannot measure the offset.** The shape term
depends only on the cycle count, so two conditions compared at the *same*
cycle count carry the same shape and it cancels from their ratio exactly.
Only a contrast spanning cycle counts says anything about `n`.
""" """
oscillator = oscillator_hz(cap, cal, period_sd) oscillator = oscillator_hz(cap, cal, period_sd)
ratio = oscillator.value / cal.reference_oscillator_hz ratio = oscillator.value / cal.reference_oscillator_hz
@ -562,10 +474,9 @@ def write_calibrated(cap, path, source, cal=None, factor=None,
class Calibrated: class Calibrated:
"""A capture already converted to nanotesla, read back from disk. """A capture already converted to nanotesla, read back from disk.
Deliberately not a `capture.Capture`: the raw counts and the host clock are A separate type from `capture.Capture`, since the raw counts and the host
gone, so the time base cannot be re-fitted and `restrict()` would be a lie. clock are gone and the time base is fixed at what was written. What survives
What survives is what a plot or a spectrum needs, plus the provenance of how is what a plot or a spectrum needs, plus the provenance of how it got there.
it got that way.
""" """
def __init__(self, path, meta, sample_index, elapsed, x, y, z, total): def __init__(self, path, meta, sample_index, elapsed, x, y, z, total):
@ -602,19 +513,8 @@ class Calibrated:
def load_calibrated(path): def load_calibrated(path):
"""Read a file written by write_calibrated().""" """Read a file written by write_calibrated()."""
meta, rows = {}, []
with open(path, newline="") as handle: with open(path, newline="") as handle:
while True: meta = capture.parse_header(handle)
position = handle.tell()
line = handle.readline()
if not line:
break
if not line.startswith("#"):
handle.seek(position)
break
key, _, value = line[1:].partition(":")
if value:
meta[key.strip()] = value.strip()
rows = list(csv.DictReader(handle)) rows = list(csv.DictReader(handle))
version = meta.get("rm3100_calibrated") version = meta.get("rm3100_calibrated")
@ -634,7 +534,9 @@ def load_calibrated(path):
if not rows: if not rows:
raise CalibrationError(f"{path}: no samples") raise CalibrationError(f"{path}: no samples")
column = lambda name: np.array([float(r[name]) for r in rows]) def column(name):
return np.array([float(r[name]) for r in rows])
return Calibrated( return Calibrated(
path, meta, path, meta,
np.array([int(r["sample_index"]) for r in rows], dtype=np.int64), np.array([int(r["sample_index"]) for r in rows], dtype=np.int64),
@ -645,9 +547,8 @@ def load_calibrated(path):
def load_any(path): def load_any(path):
"""Load a raw capture or a calibrated one, whichever this is. """Load a raw capture or a calibrated one, whichever this is.
Which format a file is, is a property of the file, not of the caller -- so Which format a file is, is a property of the file, so tools take a path and
tools take a path and get back something with axes() and a time base either get back something with axes() and a time base either way.
way, rather than each having to know.
""" """
with open(path) as handle: with open(path) as handle:
head = handle.read(4096) head = handle.read(4096)
@ -672,7 +573,7 @@ def main():
args = ap.parse_args() args = ap.parse_args()
# Check the destination before reading a capture that may be tens of # Check the destination before reading a capture that may be tens of
# megabytes, so a mistyped directory fails in a second rather than a minute. # megabytes, so a mistyped directory fails in a second.
parent = os.path.dirname(os.path.abspath(args.output)) or "." parent = os.path.dirname(os.path.abspath(args.output)) or "."
if not os.path.isdir(parent): if not os.path.isdir(parent):
sys.exit(f"{args.output}: {parent} is not a directory") sys.exit(f"{args.output}: {parent} is not a directory")
@ -694,9 +595,9 @@ def main():
print("no calibration given: converting units only, correcting nothing") print("no calibration given: converting units only, correcting nothing")
else: else:
measured = oscillator_hz(cap, cal) measured = oscillator_hz(cap, cal)
stability = rate_stability(cap) stability = ch.rate_stability(cap)
print(f"calibration {args.calibration}" print(f"calibration {args.calibration}"
+ (f" -- {cal.note}" if cal.note else "")) + (f": {cal.note}" if cal.note else ""))
print(f" rate {cap.true_rate_hz:.4f} Hz, drifting " print(f" rate {cap.true_rate_hz:.4f} Hz, drifting "
f"{stability * 1e6:.0f} ppm across the run") f"{stability * 1e6:.0f} ppm across the run")
print(f" oscillator {measured.value:,.0f} +/- {measured.sd:,.0f} Hz " print(f" oscillator {measured.value:,.0f} +/- {measured.sd:,.0f} Hz "

79
capture.py
View file

@ -1,18 +1,14 @@
"""Read an RM3100 capture and reconstruct everything the file does not store. """Read an RM3100 capture and reconstruct everything the file does not store.
A capture holds only irreducible facts -- sample count, host clock, raw counts -- A capture holds only irreducible facts, the sample count, host clock and raw
plus a header of configuration. This module turns that back into field values and counts, plus a header of configuration. This module turns that back into field
a time base, and is the single place either is derived. values and a time base, and is the single place either is derived.
Two time bases come out of `load()`: `elapsed` is `sample_index * dt_true`, where `dt_true` comes from regressing the
host clock on the sample index: the chip supplies short-term regularity, the
elapsed_nominal N * dt_nominal, using the TMRC table rate host supplies long-term rate calibration. The header's nominal rate is a model
elapsed N * dt_true, using the rate actually observed no particular part obeys, out by up to ~6% in either direction depending on
cycle count and supply, so the time base is measured instead.
The chip's RC oscillator is accurate to about +/-7% (manual section 5.2.1) and
our unit runs ~6% slow, so `elapsed_nominal` is uniform but wrongly scaled.
`dt_true` comes from regressing the host clock on the sample index: the chip
supplies short-term regularity, the host supplies long-term rate calibration.
""" """
import csv import csv
@ -22,7 +18,7 @@ import numpy as np
import rm3100 import rm3100
# Flags logger.py writes in the `warning` column, space separated. Duplicated # Flags logger.py writes in the `warning` column, space separated. Duplicated
# rather than imported so that reading a capture never pulls in the USB stack. # here so that reading a capture never pulls in the USB stack.
WARN_MISSED = "MISSED" # placeholder row: no data, keeps the index contiguous WARN_MISSED = "MISSED" # placeholder row: no data, keeps the index contiguous
WARN_AMBIGUOUS = "AMBIGUOUS" # gap ending here was of uncertain length WARN_AMBIGUOUS = "AMBIGUOUS" # gap ending here was of uncertain length
@ -32,7 +28,7 @@ class CaptureError(Exception):
class Capture: class Capture:
"""A loaded capture: raw counts, nanotesla axes, and two time bases.""" """A loaded capture: raw counts, nanotesla axes and a drift-corrected clock."""
def __init__(self, path, meta, sample_index, system_time, counts, def __init__(self, path, meta, sample_index, system_time, counts,
missed=None, ambiguous=None): missed=None, ambiguous=None):
@ -60,35 +56,26 @@ class Capture:
# Least-squares fit of host clock against grid coordinate, with both # Least-squares fit of host clock against grid coordinate, with both
# origins removed first. polyfit rescales the Vandermonde columns but # origins removed first. polyfit rescales the Vandermonde columns but
# never centres the ordinate, so fitting raw epoch seconds (~1.7e9) for # never centres the ordinate, so fitting raw epoch seconds (~1.7e9) for
# a ~4 ms slope loses six significant digits to cancellation -- measured # a ~4 ms slope loses six significant digits to cancellation. Shifting
# at 1.8e-6 of relative slope error against 3.8e-9 once centred. # each axis leaves the slope untouched and only moves the intercept.
# Subtracting a constant from each axis leaves the slope untouched and
# only moves the intercept, which nothing outside this fit uses.
index_0, time_0 = sample_index[0], system_time[0] index_0, time_0 = sample_index[0], system_time[0]
slope, intercept = np.polyfit(sample_index - index_0, slope, intercept = np.polyfit(sample_index - index_0,
system_time - time_0, 1) system_time - time_0, 1)
self.dt_true = float(slope) self.dt_true = float(slope)
# Error in the *rate*, so the sign matches the reported Hz: negative # Error in the *rate*, so the sign matches the reported Hz: negative
# means the chip samples slower than the nominal table value. # means the chip samples slower than the nominal table value. It is a
# # diagnostic of the configuration against a nominal model. calibrate.py
# Against the *nominal* model, which no particular part obeys. Its own # corrects gain against a measured per-unit reference.
# error is cycle-count dependent -- on one unit, +0.84% at cycle count
# 100 and -0.99% at 400, same supply -- so this is a diagnostic of the
# configuration, not a measurement of the oscillator, and multiplying a
# field by (1 + rate_error) is not a gain correction. calibrate.py does
# that against a measured per-unit reference.
self.rate_error = self.dt_nominal / self.dt_true - 1.0 self.rate_error = self.dt_nominal / self.dt_true - 1.0
self.residuals = (system_time - time_0) - ( self.residuals = (system_time - time_0) - (
intercept + slope * (sample_index - index_0)) intercept + slope * (sample_index - index_0))
self.elapsed_nominal = sample_index * self.dt_nominal
self.elapsed = sample_index * self.dt_true self.elapsed = sample_index * self.dt_true
# Two different things live in the timing error, and conflating them is # Two different things live in the timing error. Read latency is local:
# misleading. Read latency is local: the scatter of one read interval # the scatter of one read interval about the next. Fit residual is
# about the next. Fit residual is global: how far the whole capture # global: how far the whole capture departs from a single straight line,
# departs from a single straight line, which on a long run is dominated # which on a long run is the oscillator drifting with temperature.
# by the oscillator's rate drifting with temperature, not by the host.
self.read_jitter = float(np.diff(system_time).std()) if len(system_time) > 1 else 0.0 self.read_jitter = float(np.diff(system_time).std()) if len(system_time) > 1 else 0.0
self.residual_sd = float(self.residuals.std()) self.residual_sd = float(self.residuals.std())
self.residual_span = float(np.ptp(self.residuals)) self.residual_span = float(np.ptp(self.residuals))
@ -112,9 +99,8 @@ class Capture:
def restrict(self, start=None, end=None): def restrict(self, start=None, end=None):
"""Return a new Capture covering a window of drift-corrected seconds. """Return a new Capture covering a window of drift-corrected seconds.
Re-fits on the subset, so a window's rate is its own rather than Re-fits on the subset, so each window carries its own rate, which is
inherited -- which is what makes it usable for spotting drift across a what makes comparing windows a drift measurement.
long run by comparing windows.
""" """
keep = np.ones(len(self.sample_index), dtype=bool) keep = np.ones(len(self.sample_index), dtype=bool)
if start is not None: if start is not None:
@ -148,9 +134,6 @@ class Capture:
f"(residual sd {self.residual_sd * 1e3:.0f} ms", f"(residual sd {self.residual_sd * 1e3:.0f} ms",
f" against a {self.dt_true * 1e3:.1f} ms period)."] f" against a {self.dt_true * 1e3:.1f} ms period)."]
if self.missed.any(): if self.missed.any():
# Each lost measurement is an independent chance to insert one
# placeholder too many or too few, and the error accumulates in
# sample_index. That is the likelier cause here than thermal drift.
lines += [ lines += [
f" With {int(self.missed.sum()):,} lost measurement(s) the likely cause is " f" With {int(self.missed.sum()):,} lost measurement(s) the likely cause is "
"miscounted placeholders:", "miscounted placeholders:",
@ -179,7 +162,7 @@ class Capture:
return "\n".join(lines) return "\n".join(lines)
def _parse_header(handle): def parse_header(handle):
"""Consume leading '# key: value' lines, leaving the reader at the CSV.""" """Consume leading '# key: value' lines, leaving the reader at the CSV."""
meta = {} meta = {}
while True: while True:
@ -199,26 +182,23 @@ def _parse_header(handle):
def load(path): def load(path):
"""Load a capture written by logger.py.""" """Load a capture written by logger.py."""
with open(path, newline="") as handle: with open(path, newline="") as handle:
meta = _parse_header(handle) meta = parse_header(handle)
rows = list(csv.DictReader(handle)) rows = list(csv.DictReader(handle))
if "rm3100_capture" not in meta: if "rm3100_capture" not in meta:
raise CaptureError( raise CaptureError(
f"{path}: no capture header found. Files written before the header " f"{path}: no capture header found. Files written before the header "
"format was introduced cannot be read -- re-record them.") "format was introduced cannot be read; re-record them.")
missing = {"cycle_count", "tesla_per_count", "nominal_rate_hz"} - meta.keys() missing = {"cycle_count", "tesla_per_count", "nominal_rate_hz"} - meta.keys()
if missing: if missing:
raise CaptureError(f"{path}: header missing {', '.join(sorted(missing))}") raise CaptureError(f"{path}: header missing {', '.join(sorted(missing))}")
if len(rows) < 64: if len(rows) < 64:
raise CaptureError(f"{path}: only {len(rows)} samples, too few to analyse") raise CaptureError(f"{path}: only {len(rows)} samples, too few to analyse")
# logger.py writes "MISSED" in the count fields for a measurement it could
# not read, keeping sample_index contiguous so the chip-time grid stays
# valid across the gap.
if "warning" not in rows[0]: if "warning" not in rows[0]:
raise CaptureError( raise CaptureError(
f"{path}: no 'warning' column. Captures predating it cannot be " f"{path}: no 'warning' column. Captures predating it cannot be "
"read -- re-record them.") "read; re-record them.")
flags = [set(r["warning"].split()) for r in rows] flags = [set(r["warning"].split()) for r in rows]
missed = np.array([WARN_MISSED in f for f in flags]) missed = np.array([WARN_MISSED in f for f in flags])
# Independent of missed: a gap can be uncertain yet round to zero losses, # Independent of missed: a gap can be uncertain yet round to zero losses,
@ -228,10 +208,9 @@ def load(path):
raise CaptureError(f"{path}: every row is a lost measurement") raise CaptureError(f"{path}: every row is a lost measurement")
sample_index = np.array([int(r["sample_index"]) for r in rows], dtype=np.int64) sample_index = np.array([int(r["sample_index"]) for r in rows], dtype=np.int64)
# logger.py keeps the index contiguous even across lost measurements -- it # logger.py writes a placeholder at every grid point it misses, and stops
# writes MISSED placeholders rather than skipping grid points, and stops the # the run outright if it ever cannot, so a gap here means the file was
# run outright if it ever cannot. So a gap here means the file was truncated # truncated mid-row, damaged, or hand-edited.
# mid-row, damaged, or hand-edited, not merely cut short.
gaps = np.diff(sample_index) gaps = np.diff(sample_index)
if np.any(gaps != 1): if np.any(gaps != 1):
bad = int(sample_index[np.argmax(gaps != 1)]) bad = int(sample_index[np.argmax(gaps != 1)])
@ -245,8 +224,6 @@ def load(path):
counts = {} counts = {}
good = ~missed good = ~missed
for a in "xyz": for a in "xyz":
# Every row parses as an integer -- placeholders carry zeros, and the
# warning column is what marks them as having no data.
v = np.array([int(r[f"{a}_raw"]) for r in rows], dtype=np.float64) v = np.array([int(r[f"{a}_raw"]) for r in rows], dtype=np.float64)
if missed.any(): if missed.any():
v[missed] = np.interp(np.flatnonzero(missed), np.flatnonzero(good), v[missed] = np.interp(np.flatnonzero(missed), np.flatnonzero(good),

44
ch347.py
View file

@ -1,7 +1,7 @@
"""I2C master over a CH347 USB adapter (Waveshare USB to UART/I2C/SPI/JTAG). """I2C master over a CH347 USB adapter (Waveshare USB to UART/I2C/SPI/JTAG).
Knows nothing about any particular I2C device -- it only moves bytes to and Knows nothing about any particular I2C device; it moves bytes to and from a
from a 7-bit address. 7-bit address.
The CH347 in Mode 1 (UART1+I2C+SPI) presents three interfaces: two CDC-ACM The CH347 in Mode 1 (UART1+I2C+SPI) presents three interfaces: two CDC-ACM
interfaces that the kernel binds for the UART, and interface 2, a vendor-class interfaces that the kernel binds for the UART, and interface 2, a vendor-class
@ -37,18 +37,16 @@ CMD_IN = 0xC0 # read; low 6 bits are the byte count
CMD_SET = 0x60 # set bus speed; low bits select the rate CMD_SET = 0x60 # set bus speed; low bits select the rate
# Bus speeds, as the low nibble of CMD_SET. The fastest is the default: bus # Bus speeds, as the low nibble of CMD_SET. The fastest is the default: bus
# speed does not set the sample rate -- the sensor does -- but it does set how # speed sets how long a read takes, and so how tightly an event on the far side
# long a read takes, and so how tightly an event on the far side of the bus can # of the bus can be timestamped. The sensor sets the sample rate. Host cost per
# be timestamped. Measured host cost per sample is bus time plus ~0.6 ms of # sample is bus time plus ~0.6 ms of fixed USB round-trip latency.
# fixed USB round-trip latency: 2.13 ms at 100 kHz, 0.92 at 400, 0.79 at 750.
SPEED_20KHZ = 0 SPEED_20KHZ = 0
SPEED_100KHZ = 1 SPEED_100KHZ = 1
SPEED_400KHZ = 2 SPEED_400KHZ = 2
SPEED_750KHZ = 3 SPEED_750KHZ = 3
# The one place the rate constants are tied to their kHz figures. Callers that # The one place the rate constants are tied to their kHz figures. Callers that
# take a speed from a user or print one in a report go through this rather than # take a speed from a user or print one in a report go through this.
# keeping their own copy.
SPEEDS = { SPEEDS = {
20: SPEED_20KHZ, 20: SPEED_20KHZ,
100: SPEED_100KHZ, 100: SPEED_100KHZ,
@ -110,13 +108,13 @@ class CH347I2C:
"""Send one command stream, then read in_len status/data bytes. """Send one command stream, then read in_len status/data bytes.
Returns (data, mono, wall), the clocks read the instant the reply Returns (data, mono, wall), the clocks read the instant the reply
lands -- before any parsing or unwinding of the call stack. A caller lands, before any parsing or unwinding of the call stack. A caller
timing an event on the far side of the bus (DRDY going high, say) wants timing an event on the far side of the bus wants that moment; several
that moment, not one several Python frames later: the frames add both a Python frames later carries a systematic lag and the interpreter's
systematic lag and jitter from whatever the interpreter does between. jitter.
The stamp travels with its own reply rather than being left on the The stamp travels with its own reply, which ties it to the transaction
instance, so it cannot go stale or be picked up by the wrong call. that produced it.
""" """
written = self._dev.write(EP_OUT, bytes(out_bytes), TIMEOUT_MS) written = self._dev.write(EP_OUT, bytes(out_bytes), TIMEOUT_MS)
if written != len(out_bytes): if written != len(out_bytes):
@ -183,19 +181,15 @@ class CH347I2C:
def write_read(self, addr, data, count): def write_read(self, addr, data, count):
"""Write data then read count bytes in one transaction. """Write data then read count bytes in one transaction.
Returns (data, mono, wall) -- see _xfer() on why the stamp is returned Returns (data, mono, wall); see _xfer() on the stamp.
rather than stored.
Uses a repeated START rather than STOP-then-START, so the whole Uses a repeated START rather than STOP-then-START, so the exchange is a
exchange is a single USB round trip instead of two. Measured at single USB round trip, which halves a one-byte register read. The host
400 kHz this halves a one-byte register read (0.65 -> 0.35 ms), which round trip is what sets the sample-rate ceiling.
matters because the host round trip, not the bus, sets the sample-rate
ceiling.
The RM3100's manual draws its register reads with a STOP between the The RM3100's manual draws its register reads with a STOP between the
pointer write and the read (sections 4.5.2, 5.8.4), but the part pointer write and the read (sections 4.5.2, 5.8.4), but the part accepts
accepts a repeated START -- verified against REVID and the measurement a repeated START, verified against REVID and the measurement registers.
registers.
""" """
data = bytes(data) data = bytes(data)
if not data: if not data:
@ -213,7 +207,7 @@ class CH347I2C:
packet += [CMD_IN, CMD_STO, CMD_END] packet += [CMD_IN, CMD_STO, CMD_END]
# One status byte per clocked-out byte: the write address, the payload, # One status byte per clocked-out byte: the write address, the payload,
# and the read address -- then the payload itself. # and the read address, then the payload itself.
acks = len(data) + 2 acks = len(data) + 2
reply, mono, wall = self._xfer(packet, acks + count) reply, mono, wall = self._xfer(packet, acks + count)
if any(byte != ACK for byte in reply[:acks]): if any(byte != ACK for byte in reply[:acks]):

859
characterize.py
View file

File diff suppressed because it is too large Load diff

340
compare.py
View file

@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Compare captures taken under different conditions -- e.g. two supplies. """Compare captures taken under different conditions, e.g. two supplies.
./.venv/bin/python compare.py a.csv b.csv c.csv ... ./.venv/bin/python compare.py a.csv b.csv c.csv ...
./.venv/bin/python compare.py --group note *.csv # label from header ./.venv/bin/python compare.py --group note *.csv # label from header
@ -7,35 +7,34 @@
A label may be given as `LABEL=path`, which is the only way to identify a run A label may be given as `LABEL=path`, which is the only way to identify a run
whose capture was recorded without `--note`. Everything before the first `/` in whose capture was recorded without `--note`. Everything before the first `/` in
a label is the **condition** -- the thing under test -- and captures sharing one a label is the **condition**, the thing under test, and captures sharing one
are treated as the same configuration. `--supply LDO=3.0` then attaches a rail are treated as the same configuration. `--supply LDO=3.0` then attaches a rail
voltage to every capture in that condition. voltage to every capture in that condition.
Three lessons are built in, each learned by getting it wrong first. Three rules are built into how it reports.
**Compare |B|, not the axes.** Swapping a supply means touching the rig, and the **Compare |B|, not the axes.** Swapping a supply means touching the rig, and the
sensor does not go back exactly where it was. |B| survives a rotation; the sensor comes back somewhere close to where it was. |B| survives a rotation, so a
individual axes do not, so a per-axis difference measures the handling. The axes per-axis difference measures the handling. The axes appear here as movement
appear here only as movement diagnostics. diagnostics.
**Compare fractionally, or over a stated band.** A gain change carries the noise **Compare fractionally, or over a stated band.** A gain change carries the noise
with it, so a run reading 6% larger also reads ~6% noisier while being physically with it, so a run reading 6% larger also reads ~6% noisier while being physically
identical. And a broadband figure is only comparable over a band both captures identical. And a broadband figure is comparable over a band both captures cover,
cover -- otherwise a fast capture is scored over five times the bandwidth of a so one band is chosen for the whole invocation and printed.
slow one. One band is chosen for the whole invocation and printed.
**A magnitude difference is not automatically gain.** |B| is preserved under **A magnitude difference can be movement.** |B| is preserved under rotation, and
rotation but *not* under translation through a field gradient, so a sensor that translation through a field gradient changes it, so a sensor that moved can shift
moved can change magnitude on its own. Two diagnostics bound it: magnitude on its own. Two diagnostics bound that:
rotation angle the angle between mean field directions. rotation angle the angle between mean field directions.
gain residual what is left after the best single scale factor, as a gain residual what is left after the best single scale factor, as a
fraction of |B|. Zero means one number relates the two fraction of |B|. Zero means one number relates the two
captures, which is what a gain change looks like. captures, which is what a gain change looks like.
Neither can prove the sensor held still, so neither vetoes the comparison; they Both size the doubt; the comparison runs either way. The reproducibility floor
size the doubt. The reproducibility floor from repeats of one condition is the from repeats of one condition is the honest error bar, and this prints it when
honest error bar, and this prints it when there are repeats to use. there are repeats to use.
""" """
import argparse import argparse
@ -60,11 +59,9 @@ import rm3100
RATIO_SPREAD_OK = 0.01 # 1% RATIO_SPREAD_OK = 0.01 # 1%
ROTATION_OK_DEG = 0.5 ROTATION_OK_DEG = 0.5
# An axis whose mean field is near zero has a ratio dominated by noise -- a few # An axis whose mean field is near zero has a ratio dominated by noise, so the
# nT of drift on a 10 nT mean is a ratio of 1.3 with nothing moving. Such an axis # spread test drops it. The rotation check works on the whole vector's direction,
# is excluded from the spread test rather than allowed to dominate it. The # where a small component contributes little, and needs no such guard.
# rotation check needs no such guard: it works on the direction of the whole
# vector, where a small component simply contributes little.
RATIO_AXIS_MIN_FRACTION = 0.05 # of |B| RATIO_AXIS_MIN_FRACTION = 0.05 # of |B|
# Two captures whose rates differ by less than this are treated as the same # Two captures whose rates differ by less than this are treated as the same
@ -74,9 +71,9 @@ RATE_DISTINCT = 0.01 # 1%
# Condition and variant are separated by this in a label: `LDO/cc100`. # Condition and variant are separated by this in a label: `LDO/cc100`.
CONDITION_SEP = "/" CONDITION_SEP = "/"
# Hue carries the condition -- the thing under test -- and never the cycle # Hue carries the condition, the thing under test, and the cycle count goes to
# count, so a chart never repaints when a run is added. Validated all-pairs in # line style below, so a chart keeps its colours when a run is added. Validated
# light mode against the #fcfcfb surface: worst CVD dE 24.7, normal-vision 33.6. # all-pairs in light mode against the #fcfcfb surface.
CONDITION_COLORS = ["#2a78d6", "#eb6834", "#1baf7a"] CONDITION_COLORS = ["#2a78d6", "#eb6834", "#1baf7a"]
# Cycle count is the secondary encoding, carried by line style and marker, so # Cycle count is the secondary encoding, carried by line style and marker, so
# identity never rests on colour alone. # identity never rests on colour alone.
@ -91,10 +88,9 @@ VARIANT_MARKERS = ["o", "s", "^", "D"]
def split_label(argument): def split_label(argument):
"""Split `LABEL=path` into (label, path); (None, path) when unlabelled. """Split `LABEL=path` into (label, path); (None, path) when unlabelled.
A path may itself contain '=', so a bare existing path always wins over the A path may itself contain '=', so a bare existing path wins over the
labelled reading. When neither reading names a file that exists, the labelled reading. When neither reading names a file that exists, the
labelled one is returned so the error message quotes the path the user labelled one is returned, which puts the likelier path in the error message.
most likely meant.
""" """
label, separator, path = argument.partition("=") label, separator, path = argument.partition("=")
if not separator: if not separator:
@ -160,9 +156,8 @@ def variant_of(label):
def conditions_in_order(records): def conditions_in_order(records):
"""Distinct conditions, in the order they were given on the command line. """Distinct conditions, in the order they were given on the command line.
Command-line order, not alphabetical: it is the one thing the caller Command-line order is the one thing the caller controls, so contrasts read
controls, so contrasts read in the direction they asked for and a colour in the direction they asked for and a colour stays put across runs.
never moves between runs because a condition was renamed.
""" """
seen = [] seen = []
for label, _, _ in records: for label, _, _ in records:
@ -186,15 +181,14 @@ def summarise(cap, band):
for key, _, _ in ch.SERIES: for key, _, _ in ch.SERIES:
v = d[key] v = d[key]
median_asd, band_rms, peak, peak_hz = ch.band_stats(v, fs, band) median_asd, band_rms, peak, peak_hz = ch.band_stats(v, fs, band)
# The absolute figures are nanotesla, and nanotesla are what a gain
# change moves, so every cross-condition comparison uses the columns
# divided by that condition's own |B|.
out[key] = { out[key] = {
"sd": v.std(), "sd": v.std(),
"white": ch.white_sd(v), "white": ch.white_sd(v),
"ppm": v.std() / field * 1e6, "ppm": v.std() / field * 1e6,
"asd": median_asd, "asd": median_asd,
# Every absolute figure above is in nanotesla, and nanotesla are
# what a gain change moves. A condition that reads 7% smaller reads
# 7% quieter in nT while being no quieter at all, so any comparison
# across conditions has to divide by that condition's own |B|.
"asd_ppm": median_asd / field * 1e6, "asd_ppm": median_asd / field * 1e6,
"white_ppm": ch.white_sd(v) / field * 1e6, "white_ppm": ch.white_sd(v) / field * 1e6,
"band_rms": band_rms, "band_rms": band_rms,
@ -218,11 +212,9 @@ def axis_ratio_spread(a, b, ratio):
"""Spread of the per-axis scale ratios, or None if it cannot be measured. """Spread of the per-axis scale ratios, or None if it cannot be measured.
A pure gain change scales X, Y and Z identically, so the spread is the A pure gain change scales X, Y and Z identically, so the spread is the
movement test. It is only meaningful on axes that actually carry field: movement test. Dividing two near-zero means amplifies their noise without
dividing two near-zero means amplifies their noise without bound, and one bound, so axes below RATIO_AXIS_MIN_FRACTION of |B| in *either* capture are
such axis is enough to report a stationary sensor as moved. Axes below dropped, and with fewer than two left the test abstains.
RATIO_AXIS_MIN_FRACTION of |B| in *either* capture are therefore dropped,
and with fewer than two left the test abstains.
""" """
usable = [i for i in range(3) usable = [i for i in range(3)
if abs(a["mean"][i]) >= RATIO_AXIS_MIN_FRACTION * a["field"] if abs(a["mean"][i]) >= RATIO_AXIS_MIN_FRACTION * a["field"]
@ -237,10 +229,10 @@ def gain_and_movement(a, b):
"""(scale, residual, rotation deg) relating two mean field vectors. """(scale, residual, rotation deg) relating two mean field vectors.
`scale` is the single factor that best maps a onto b, and `residual` is what `scale` is the single factor that best maps a onto b, and `residual` is what
that factor cannot explain, as a fraction of |b|. A pure gain change leaves that factor leaves over, as a fraction of |b|. A pure gain change leaves a
a residual of zero at any angle; a rotation leaves a large one. Unlike a residual of zero at any angle; a rotation leaves a large one. It stays finite
per-axis ratio this stays finite when an axis passes through zero, which is when an axis passes through zero, which is when a re-oriented sensor most
exactly when the sensor has been re-oriented and the diagnostic is needed. needs the diagnostic.
""" """
ma, mb = a["mean"], b["mean"] ma, mb = a["mean"], b["mean"]
scale = float(ma @ mb / (ma @ ma)) scale = float(ma @ mb / (ma @ ma))
@ -257,42 +249,36 @@ def fit_rate_model(caps):
linear in cycle count with slope 1/C and intercept h. Two cycle counts linear in cycle count with slope 1/C and intercept h. Two cycle counts
determine both exactly; more are a least-squares fit. determine both exactly; more are a least-squares fit.
Fitting it per condition is what separates an oscillator that changed from a Fitting it per condition separates an oscillator that changed from a model
model that was never right: a supply that only scales the clock moves C and that was wrong to begin with: a supply that only scales the clock moves C and
h together by one factor, leaving the *shape* of the model alone. h together by one factor, leaving the *shape* of the model alone.
""" """
try: try:
# The uncertainty on each period is how far the oscillator drifted # The uncertainty on each period is how far the oscillator drifted
# during that capture; the worst of them bounds the fit. # during that capture; the worst of them bounds the fit.
drift = max(calibrate.rate_stability(c) for c in caps) drift = max(ch.rate_stability(c) for c in caps)
fit = calibrate.fit_timing([c.cycle_count for c in caps], return calibrate.fit_timing([c.cycle_count for c in caps],
[c.dt_true for c in caps], [c.dt_true for c in caps],
period_sd=drift or 1e-9) period_sd=drift or 1e-9)
return fit
except calibrate.CalibrationError as exc: except calibrate.CalibrationError as exc:
# One fit, in calibrate.py, so sweep.py writing a calibration and this # Translated because callers here already handle ValueError. The fit
# reporting one cannot drift apart. The exception type is translated # itself stays in calibrate.py, shared with sweep.py.
# because callers here already handle ValueError.
raise ValueError(str(exc)) from exc raise ValueError(str(exc)) from exc
def decimation_paths(fast, slow, k, band): def decimation_paths(fast, slow, k, band):
"""Noise of a fast capture decimated by k, against a natively slow one. """Noise of a fast capture decimated by k, against a natively slow one.
The question this answers is whether sampling fast and filtering down is Answers whether sampling fast and filtering down is equivalent to having
equivalent to having sampled slowly -- which is the whole basis for sampled slowly, which is the basis for recommending a low cycle count. Both
recommending a low cycle count. Both decimations are shown because they decimations are shown: the boxcar is what the sensor's own integration does,
differ in exactly one way that matters: the boxcar is what the sensor's own poor stopband and all, and the FIR rejects what the boxcar folds in.
integration does, poor stopband and all, while the FIR is what a filter
designed for the job does, and only the second actually rejects what would
otherwise fold in.
Compare the ASD column, not the sd column. Each path has a different Compare the ASD column, which is quoted over a common band. Each path has
effective noise bandwidth, so their per-sample sd figures are not on the its own effective noise bandwidth, which the per-sample sd carries with it.
same footing, while ASD over a common band is.
""" """
v_fast = dict(fast["cap"].axes())["total"] v_fast = fast["axes"]["total"]
v_slow = dict(slow["cap"].axes())["total"] v_slow = slow["axes"]["total"]
fs_fast, fs_slow = fast["fs"], slow["fs"] fs_fast, fs_slow = fast["fs"], slow["fs"]
rows = [("raw fast", v_fast, fs_fast, fast["field"])] rows = [("raw fast", v_fast, fs_fast, fast["field"])]
for method in ("boxcar", "fir"): for method in ("boxcar", "fir"):
@ -302,9 +288,9 @@ def decimation_paths(fast, slow, k, band):
out = [] out = []
for name, v, fs, field in rows: for name, v, fs, field in rows:
median_asd, band_rms, _, _ = ch.band_stats(v, fs, band) median_asd, band_rms, _, _ = ch.band_stats(v, fs, band)
# The two cycle counts have their own gains, so the fractional column
# is the comparable one here too.
out.append({"name": name, "fs": fs, "asd": median_asd, out.append({"name": name, "fs": fs, "asd": median_asd,
# The two cycle counts do not share a gain either, so the
# fractional column is the comparable one here too.
"asd_ppm": median_asd / field * 1e6, "asd_ppm": median_asd / field * 1e6,
"band_rms": band_rms, "white": ch.white_sd(v), "band_rms": band_rms, "white": ch.white_sd(v),
"lines": ch.sample_locked_lines(v)}) "lines": ch.sample_locked_lines(v)})
@ -316,8 +302,8 @@ def decimation_pairs(entries):
`entries` are (label, summary-or-capture-like) with .cycle_count and a rate. `entries` are (label, summary-or-capture-like) with .cycle_count and a rate.
A pair qualifies when one cycle count is an integer multiple of the other, A pair qualifies when one cycle count is an integer multiple of the other,
since that is when averaging k samples of the fast run buys the same which is when averaging k samples of the fast run buys the same integration
integration time as one sample of the slow one. time as one sample of the slow one.
""" """
by_condition = defaultdict(list) by_condition = defaultdict(list)
for label, item in entries: for label, item in entries:
@ -338,13 +324,12 @@ def decimation_pairs(entries):
def comparable_rates(loaded): def comparable_rates(loaded):
"""Every rate that will be scored, decimated paths included. """Every rate that will be scored, decimated paths included.
The band has to fit inside the narrowest of these, not merely inside the The band has to fit inside the narrowest of these. Decimating by k lands at
slowest capture: decimating by k lands at fs/k, and the chip's per-axis fs/k, and the per-axis overhead puts that a little *below* the natively-slow
overhead means that is a little *below* the natively-slow rate rather than rate; two percent of bandwidth is enough to score the decimated path on its
equal to it. Two percent of bandwidth is enough to score the decimated path own filter rolloff.
on its own filter rolloff and report it as the quieter one.
Runs before the summaries exist, so it takes captures rather than records. Runs before the summaries exist, so it takes captures.
""" """
entries = [(label, {"fs": cap.true_rate_hz, entries = [(label, {"fs": cap.true_rate_hz,
"cycle_count": cap.cycle_count}) "cycle_count": cap.cycle_count})
@ -359,9 +344,8 @@ def locked_line_verdict(records):
A field line sits at a fixed frequency, so it lands on a different fraction A field line sits at a fixed frequency, so it lands on a different fraction
of fs when the rate changes. One that keeps the same fraction is locked to of fs when the rate changes. One that keeps the same fraction is locked to
the sampling and is an artefact of measuring rather than something measured. the sampling, and so is an artifact of measuring. Deciding this needs two
Deciding this needs two captures at genuinely different rates, which is why captures at genuinely different rates, which is why it lives here.
it lives here and not in characterize.py.
""" """
seen = defaultdict(list) seen = defaultdict(list)
for label, _, s in records: for label, _, s in records:
@ -388,7 +372,7 @@ def print_overview(records, band, supplies):
for label, _, s in records: for label, _, s in records:
t = s["total"] t = s["total"]
print(f"{label[:20]:<20} {s['cycle_count']:5d} {s['fs']:9.3f} " print(f"{label[:20]:<20} {s['cycle_count']:5d} {s['fs']:9.3f} "
f"{calibrate.rate_stability(s['cap']) * 1e6:6.0f}p " f"{ch.rate_stability(s['cap']) * 1e6:6.0f}p "
f"{s['field']:10,.0f} {t['sd']:7.1f} {t['white']:7.2f} " f"{s['field']:10,.0f} {t['sd']:7.1f} {t['white']:7.2f} "
f"{t['asd']:6.2f} {s['allan'][0]:6.2f} | " f"{t['asd']:6.2f} {s['allan'][0]:6.2f} | "
f"{t['white_ppm']:7.1f} {t['asd_ppm']:7.1f}") f"{t['white_ppm']:7.1f} {t['asd_ppm']:7.1f}")
@ -396,17 +380,16 @@ def print_overview(records, band, supplies):
f"over {band[0]:g}-{band[1]:.1f} Hz.") f"over {band[0]:g}-{band[1]:.1f} Hz.")
print(" Right of the bar, the same two divided by that capture's own |B|: " print(" Right of the bar, the same two divided by that capture's own |B|: "
"ppm and ppm/rtHz.") "ppm and ppm/rtHz.")
print(" Compare conditions on the right. A condition whose gain differs " print(" Compare conditions on the right, where each capture is divided by "
"records proportionally\n smaller nanotesla of noise while being no " "its own |B|, so a\n gain difference stays out of the noise figure. "
"quieter, so the absolute columns would\n credit a low gain as a " "Use the left against Table 3-1,\n which is quoted in nT at one "
"low noise floor. Use the left against Table 3-1, which is\n " "supply.")
"quoted in nT at one supply.") print(" white = sd(diff)/sqrt(2), which rejects drift. "
print(" white = sd(diff)/sqrt(2), which rejects drift; sd does not. "
"Allan = best sigma by averaging.") "Allan = best sigma by averaging.")
print(" drift is how far the oscillator moved across the run, in ppm -- the " print(" drift is how far the oscillator moved across the run, in ppm: the "
"error bar on that\n capture's rate, and on anything derived from it. " "error bar on that\n capture's rate, and on anything derived from it. "
"A tenth of a percent is ordinary\n for an LR relaxation oscillator. " "A tenth of a percent is ordinary\n for an LR relaxation oscillator. "
"It is a timing figure: nothing magnetic reaches it.") "It is a timing figure.")
if supplies: if supplies:
rails = " ".join(f"{c} {v:g} V" for c, v in sorted(supplies.items())) rails = " ".join(f"{c} {v:g} V" for c, v in sorted(supplies.items()))
print(f" rails: {rails}") print(f" rails: {rails}")
@ -431,8 +414,7 @@ def print_rate_model(records):
f"{fit.axis_overhead_s * 1e6:6.2f} +/- " f"{fit.axis_overhead_s * 1e6:6.2f} +/- "
f"{fit.axis_overhead_s_sd * 1e6:4.2f} us " f"{fit.axis_overhead_s_sd * 1e6:4.2f} us "
f"({fit.points} points, " f"({fit.points} points, "
+ ("no residual: two points fit both terms exactly, so a bad " + ("exactly determined, so no residual)"
"point cannot show up)"
if fit.points == 2 else if fit.points == 2 else
f"residual {fit.residual_ppm:.0f} ppm)")) f"residual {fit.residual_ppm:.0f} ppm)"))
print(f" {'rm3100.py':<12} C {rm3100.COUNTS_PER_SECOND:10,.0f} counts/s " print(f" {'rm3100.py':<12} C {rm3100.COUNTS_PER_SECOND:10,.0f} counts/s "
@ -468,16 +450,15 @@ def print_locked_lines(records):
if distinct: if distinct:
print(" -> SAMPLE-LOCKED. It holds this fraction of fs across " print(" -> SAMPLE-LOCKED. It holds this fraction of fs across "
"captures whose rates\n differ, so it moves with the " "captures whose rates\n differ, so it moves with the "
"sampling and is not a field at a fixed\n frequency. " "sampling. Decimating a faster capture\n removes it.")
"Decimating a faster capture avoids it; filtering cannot.")
elif len(hits) > 1: elif len(hits) > 1:
print(" -> seen in several captures, but at rates too close " print(" -> seen in several captures, at rates too close together "
"together to tell a\n sample-locked line from a fixed " "to tell a\n sample-locked line from a fixed one. "
"one. Re-run with rates further apart.") "Re-run with rates further apart.")
else: else:
print(" -> seen in one capture only, so nothing here says " print(" -> seen in one capture only. Deciding whether it is "
"whether it is locked to\n the sampling or to a " "locked to the sampling\n or to a frequency needs a "
"frequency.") "second capture at another rate.")
def print_decimation(records, band): def print_decimation(records, band):
@ -503,27 +484,25 @@ def print_decimation(records, band):
raw = rows[0] raw = rows[0]
fir = next(r for r in rows if r["name"].startswith("fir")) fir = next(r for r in rows if r["name"].startswith("fir"))
native = rows[-1] native = rows[-1]
# Two ratios, because one is clean and the other is not. Decimating a # Two ratios. Decimating leaves the gain and the sensor position
# capture cannot change its gain or move the sensor, so the first is a # untouched, so the first measures the filter alone; the second brings
# measurement of the filter alone. The second brings in a whole separate # in a separate run, carrying every difference between the two.
# run, and carries every difference between the two runs with it.
print(f" -> decimating this capture changes its own floor by " print(f" -> decimating this capture changes its own floor by "
f"{(fir['asd_ppm'] / raw['asd_ppm'] - 1) * 100:+.1f}% " f"{(fir['asd_ppm'] / raw['asd_ppm'] - 1) * 100:+.1f}%, "
f"-- no other difference is involved.") f"this capture against itself.")
print(f" Against the separately-recorded slow capture it is " print(f" Against the separately-recorded slow capture it is "
f"{fir['asd_ppm'] / native['asd_ppm']:.3f}x, which also carries " f"{fir['asd_ppm'] / native['asd_ppm']:.3f}x, which also carries "
f"whatever\n differs between two runs.") f"whatever\n differs between two runs.")
print("\n Compare the ASD ppm column: each path has a different effective " print("\n Compare the ASD ppm column, which is quoted over a common band "
"noise bandwidth,\n so per-sample sd figures are not on the same " "and divided by\n each capture's own |B|. Each path carries its own "
"footing, and the two cycle counts do\n not share a gain, so " "effective noise bandwidth in\n the per-sample sd, and its own gain "
"absolute nT are not either.") "in the absolute nT.")
# The candidate explanations for a gain that moves with the supply, in the # The candidate explanations for a gain that moves with the supply, in the
# order they are reported. `both` is included because it is the obvious thing # order reported. `both` stacks the two, and its residual shows what that costs:
# to try and because it is instructive when it fails: the rate change *is* the # the rate change *is* the supply change seen through the oscillator, so the two
# supply change seen through the oscillator, so applying the two on top of each # together count the same physics twice.
# other counts the same physics twice.
CORRECTIONS = [("raw", "raw"), CORRECTIONS = [("raw", "raw"),
("rate", "x measured fs"), ("rate", "x measured fs"),
("supply", "x rail volts"), ("supply", "x rail volts"),
@ -533,11 +512,11 @@ CORRECTIONS = [("raw", "raw"),
def correction_names(cal): def correction_names(cal):
"""Row labels, naming what the rate row is actually referenced to. """Row labels, naming what the rate row is actually referenced to.
Uncalibrated it is `1 + rate_error`, which measures the rate against Uncalibrated it is `1 + rate_error`, measured against rm3100's *nominal*
rm3100's *nominal* model -- and that model misfits by a cycle-count model, whose misfit is cycle-count dependent, so the row carries meaning
dependent amount, so the row means something only inside a matched inside a matched cycle-count contrast, where that misfit cancels. With a
cycle-count contrast. With a calibration it is the measured oscillator calibration it is the measured oscillator against a stated reference, which
against a stated reference, which means the same thing everywhere. means the same thing everywhere.
""" """
renamed = {"rate": "x oscillator", "both": "x osc x volts"} renamed = {"rate": "x oscillator", "both": "x osc x volts"}
return [(key, renamed.get(key, name) if cal else name) return [(key, renamed.get(key, name) if cal else name)
@ -549,16 +528,12 @@ def corrected_fields(s, supplies, reference_volts, cal=None):
rate the chip's own clock is measured every run, so a gain that tracks rate the chip's own clock is measured every run, so a gain that tracks
the clock is correctable from the capture alone. With a calibration the clock is correctable from the capture alone. With a calibration
this is the measured oscillator against a stated reference. Without this is the measured oscillator against a stated reference; without
one it falls back to `1 + rate_error`, which references rm3100's one it falls back to `1 + rate_error`, which carries meaning inside
nominal model -- and that model's error is cycle-count dependent a matched cycle-count contrast.
(+0.84% at cc=100, -0.99% at cc=400 on one unit, same supply), so supply a gain simply ratiometric with the rail. Needs the rail measured,
the fallback is only meaningful inside a matched cycle-count and nothing from the capture.
contrast, where it cancels. both the two together, which holds if they are independent mechanisms.
supply a gain that is simply ratiometric with the rail needs the rail
measured, but nothing from the capture.
both the two together, which is only right if they are independent
mechanisms rather than one seen twice.
""" """
raw = s["field"] raw = s["field"]
if cal is not None: if cal is not None:
@ -574,7 +549,7 @@ def corrected_fields(s, supplies, reference_volts, cal=None):
def print_contrasts(records, supplies, cal=None): def print_contrasts(records, supplies, cal=None):
"""Condition contrasts at matched cycle count -- the designed comparison.""" """Condition contrasts at matched cycle count: the designed comparison."""
conditions = conditions_in_order(records) conditions = conditions_in_order(records)
if len(conditions) < 2: if len(conditions) < 2:
return return
@ -608,10 +583,9 @@ def print_contrasts(records, supplies, cal=None):
f"{right[key] - left[key]:+9,.1f} nT " f"{right[key] - left[key]:+9,.1f} nT "
f"{(right[key]/left[key] - 1) * 100:+6.2f}%") f"{(right[key]/left[key] - 1) * 100:+6.2f}%")
contrasts[(ca, cb)].append((cc, math.log(right["raw"] / left["raw"]))) contrasts[(ca, cb)].append((cc, math.log(right["raw"] / left["raw"])))
# Rather than only testing whether gain goes as 1/V, fit the power # Fit the power gain actually goes as. -1 is exactly ratiometric,
# it actually goes as. -1 is exactly ratiometric, 0 is no supply # 0 is no supply dependence. The rate gets the same treatment: same
# dependence at all, and the rate gets the same treatment because # oscillator, seen from the other end.
# both are the same oscillator seen from different ends.
va, vb = supplies.get(ca), supplies.get(cb) va, vb = supplies.get(ca), supplies.get(cb)
if va and vb and va != vb: if va and vb and va != vb:
dv = math.log(vb / va) dv = math.log(vb / va)
@ -619,17 +593,25 @@ def print_contrasts(records, supplies, cal=None):
(cc, math.log(right["raw"] / left["raw"]) / dv, (cc, math.log(right["raw"] / left["raw"]) / dv,
math.log(b["fs"] / a["fs"]) / dv)) math.log(b["fs"] / a["fs"]) / dv))
if cal is not None:
print("\n The x oscillator rows above hold for any "
"gain_offset_counts, including the\n "
f"{cal.gain_offset_counts:+.2f} this calibration carries. The "
"shape term depends only on the cycle\n count, so at a matched "
"cycle count both conditions carry the same one and it\n "
"cancels. A contrast spanning cycle counts is what measures the "
"offset.")
for (ca, cb), points in exponents.items(): for (ca, cb), points in exponents.items():
va, vb = supplies[ca], supplies[cb] va, vb = supplies[ca], supplies[cb]
print(f"\n how {ca} -> {cb} ({va:g} V -> {vb:g} V) scales with the rail:") print(f"\n how {ca} -> {cb} ({va:g} V -> {vb:g} V) scales with the rail:")
for cc, field_power, rate_power in points: for cc, field_power, rate_power in points:
print(f" cycle count {cc}: |B| ~ V^{field_power:+.3f} " print(f" cycle count {cc}: |B| ~ V^{field_power:+.3f} "
f"rate ~ V^{rate_power:+.3f}") f"rate ~ V^{rate_power:+.3f}")
print(" V^-1 on |B| would be exactly ratiometric -- gain simply " print(" V^-1 on |B| is exactly ratiometric, gain proportional to "
"proportional to the\n rail. V^0 would be no supply " "the rail; V^0 is\n no supply dependence. Read the residual "
"dependence. Read the residual of the `x rail\n volts` row " "of the `x rail volts` row above\n against the "
"above against the reproducibility figure below before " "reproducibility figure below to see what these support.")
"believing\n either endpoint.")
for (ca, cb), points in contrasts.items(): for (ca, cb), points in contrasts.items():
if len(points) < 2: if len(points) < 2:
@ -639,11 +621,11 @@ def print_contrasts(records, supplies, cal=None):
f"mean {np.mean(values) * 100:+.2f}%, " f"mean {np.mean(values) * 100:+.2f}%, "
f"spread {(max(values) - min(values)) * 100:.2f}%") f"spread {(max(values) - min(values)) * 100:.2f}%")
print(" A small spread means the condition acts the same at every " print(" A small spread means the condition acts the same at every "
"cycle count, which\n is what a scale factor does and what an " "cycle count, which\n is the signature of a scale factor. An "
"environmental step does not.") "environmental step spreads wider.")
# A condition measured twice bounds everything this experiment cannot # A condition measured twice bounds everything this experiment cannot
# separate -- handling, gradient, drift -- and is the only honest error bar. # separate, handling and gradient and drift, and is the honest error bar.
print("\n reproducibility within one condition (the error bar on all of " print("\n reproducibility within one condition (the error bar on all of "
"the above):") "the above):")
any_repeat = False any_repeat = False
@ -659,20 +641,18 @@ def print_contrasts(records, supplies, cal=None):
+ f" spread {(max(fields)/min(fields) - 1) * 100:.2f}%") + f" spread {(max(fields)/min(fields) - 1) * 100:.2f}%")
if any_repeat: if any_repeat:
print(" These repeats differ in cycle count as well, so the spread " print(" These repeats differ in cycle count as well, so the spread "
"is an upper bound\n on run-to-run reproducibility, not a " "is an upper bound\n on run-to-run reproducibility.")
"clean measurement of it.")
else: else:
print(" none -- no condition was captured twice, so there is no " print(" none: no condition was captured twice, so there is no error "
"error bar at all.") "bar at all.")
def print_pairwise(records): def print_pairwise(records):
if len(records) < 2: if len(records) < 2:
return return
print("\n=== pairwise: |B|, and how far the sensor moved ===") print("\n=== pairwise: |B|, and how far the sensor moved ===")
print("|B| survives a rotation, so a difference is only movement if the " print("|B| survives a rotation, so a difference is movement where the "
"sensor also\ntranslated through a gradient. These size that doubt; " "sensor also\ntranslated through a gradient. These size that doubt.")
"they do not veto the result.")
for (la, _, a), (lb, _, b) in itertools.combinations(records, 2): for (la, _, a), (lb, _, b) in itertools.combinations(records, 2):
ratio = b["mean"] / a["mean"] ratio = b["mean"] / a["mean"]
spread = axis_ratio_spread(a, b, ratio) spread = axis_ratio_spread(a, b, ratio)
@ -696,8 +676,8 @@ def print_pairwise(records):
moved = angle > ROTATION_OK_DEG or residual > RATIO_SPREAD_OK moved = angle > ROTATION_OK_DEG or residual > RATIO_SPREAD_OK
if moved: if moved:
print(" -> the sensor moved between these two, so some of the " print(" -> the sensor moved between these two, so some of the "
"|B| difference may be\n a different place in the field " "|B| difference may be\n a different place in the "
"rather than a different scale factor.") "field.")
else: else:
print(" -> the sensor held still, so the |B| difference is a " print(" -> the sensor held still, so the |B| difference is a "
"scale change.") "scale change.")
@ -707,34 +687,13 @@ def print_pairwise(records):
# Figure # Figure
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------
def style(ax):
ax.set_facecolor(ch.SURFACE)
ax.grid(True, which="both", color=ch.GRID, linewidth=0.7)
ax.set_axisbelow(True)
for side in ("top", "right"):
ax.spines[side].set_visible(False)
for side in ("left", "bottom"):
ax.spines[side].set_color(ch.GRID)
ax.tick_params(colors=ch.TEXT_SECONDARY, labelsize=9, length=0)
ax.xaxis.label.set_color(ch.TEXT_SECONDARY)
ax.yaxis.label.set_color(ch.TEXT_SECONDARY)
def title(ax, text):
ax.set_title(text, loc="left", color=ch.TEXT_PRIMARY, fontsize=12,
fontweight="bold", pad=8)
def binned_asd(freqs, asd, bins=140): def binned_asd(freqs, asd, bins=140):
"""Median ASD in log-spaced frequency bins, for a trace that can be read. """Median ASD in log-spaced frequency bins, for a trace that can be read.
A raw periodogram at these lengths draws tens of thousands of points into a A raw periodogram at these lengths draws tens of thousands of points into a
few hundred pixels, which renders as a solid band whose apparent thickness few hundred pixels and renders as a solid band whose thickness is chi-square
is chi-square scatter rather than anything about the sensor. Binning by the scatter. Binning by the median keeps the floor where the quoted numbers put
median keeps the floor exactly where the quoted numbers put it -- they are it, those being medians too, and keeps a line a line.
medians too -- while a line stays a line. Narrow lines survive because a
log bin up here is many raw bins wide only at high frequency, where these
captures have no lines.
""" """
edges = np.geomspace(freqs[0], freqs[-1], bins + 1) edges = np.geomspace(freqs[0], freqs[-1], bins + 1)
index = np.digitize(freqs, edges) - 1 index = np.digitize(freqs, edges) - 1
@ -760,28 +719,28 @@ def make_figure(records, band, supplies, path, cal=None):
fig, axs = plt.subplots(2, 2, figsize=(13.5, 9), dpi=150) fig, axs = plt.subplots(2, 2, figsize=(13.5, 9), dpi=150)
fig.patch.set_facecolor(ch.SURFACE) fig.patch.set_facecolor(ch.SURFACE)
for ax in axs.flat: for ax in axs.flat:
style(ax) ch.style(ax)
a = axs[0, 0] a = axs[0, 0]
a.axvspan(band[0], band[1], color=ch.GRID, alpha=0.55, zorder=0, a.axvspan(band[0], band[1], color=ch.GRID, alpha=0.55, zorder=0,
linewidth=0) linewidth=0)
for label, _, s in records: for label, _, s in records:
freqs, asd = binned_asd(*ch.welch_asd(dict(s["cap"].axes())["total"], s["fs"], freqs, asd = binned_asd(*ch.welch_asd(
nperseg=ch.segment_length(s["fs"]))) s["axes"]["total"], s["fs"], nperseg=ch.segment_length(s["fs"])))
# Fractional, because the conditions do not share a gain. In absolute # Fractional, because each condition has its own gain: in absolute
# nT/rtHz a condition reading 7% smaller plots 7% lower while being no # nT/rtHz a condition reading 7% smaller plots 7% lower while being
# quieter, and the chart would say the opposite of the truth. # equally quiet.
asd = asd / s["field"] * 1e6 asd = asd / s["field"] * 1e6
a.loglog(freqs, asd, color=color[condition_of(label)], a.loglog(freqs, asd, color=color[condition_of(label)],
linestyle=dash[variant_of(label)], linewidth=1.4, alpha=0.9, linestyle=dash[variant_of(label)], linewidth=1.4, alpha=0.9,
label=label) label=label)
a.set_xlim(*ch.FRACTIONAL_ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM) a.set_xlim(*ch.FRACTIONAL_ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM)
ch.decade_ticks(a) ch.decade_ticks(a)
title(a, "Spectral density of |B|, as a fraction of |B|") ch.title(a, "Spectral density of |B|, as a fraction of |B|")
a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz") a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz")
a.annotate(f"shaded: {band[0]:g}–{band[1]:.0f} Hz, the compared band." a.annotate(f"shaded: {band[0]:g}–{band[1]:.0f} Hz, the compared band."
"\nnormalised by each capture's own |B|, so a gain" "\nnormalised by each capture's own |B|, which keeps"
"\ndifference is not read as a noise difference", "\na gain difference out of the noise figure",
xy=(0.02, 0.05), xycoords="axes fraction", xy=(0.02, 0.05), xycoords="axes fraction",
color=ch.TEXT_SECONDARY, fontsize=9) color=ch.TEXT_SECONDARY, fontsize=9)
a.legend(frameon=False, fontsize=9, labelcolor=ch.TEXT_SECONDARY, a.legend(frameon=False, fontsize=9, labelcolor=ch.TEXT_SECONDARY,
@ -789,7 +748,7 @@ def make_figure(records, band, supplies, path, cal=None):
a = axs[0, 1] a = axs[0, 1]
plotted = _plot_decimation(a, records, band, color, dash) plotted = _plot_decimation(a, records, band, color, dash)
title(a, "Fast capture, filtered and decimated") ch.title(a, "Fast capture, filtered and decimated")
a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz") a.set_xlabel("frequency (Hz)"); a.set_ylabel("ppm of |B| per √Hz")
if plotted: if plotted:
a.set_xlim(*ch.FRACTIONAL_ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM) a.set_xlim(*ch.FRACTIONAL_ASD_HZ); a.set_ylim(*ch.FRACTIONAL_ASD_PPM)
@ -810,7 +769,7 @@ def make_figure(records, band, supplies, path, cal=None):
linestyle=dash[variant_of(label)], linewidth=1.6, label=label) linestyle=dash[variant_of(label)], linewidth=1.6, label=label)
a.set_xlim(*ch.ALLAN_TAU_S); a.set_ylim(*ch.FRACTIONAL_ALLAN_PPM) a.set_xlim(*ch.ALLAN_TAU_S); a.set_ylim(*ch.FRACTIONAL_ALLAN_PPM)
ch.decade_ticks(a) ch.decade_ticks(a)
title(a, "Allan deviation of |B|, as a fraction of |B|") ch.title(a, "Allan deviation of |B|, as a fraction of |B|")
a.set_xlabel("averaging time τ (s)"); a.set_ylabel("σ (ppm of |B|)") a.set_xlabel("averaging time τ (s)"); a.set_ylabel("σ (ppm of |B|)")
a.annotate("slope −½ = white noise; upturn = drift", a.annotate("slope −½ = white noise; upturn = drift",
xy=(0.02, 0.04), xycoords="axes fraction", xy=(0.02, 0.04), xycoords="axes fraction",
@ -840,18 +799,18 @@ def _plot_decimation(ax, records, band, color, dash):
for condition, (_, fast), (_, slow), k in decimation_pairs( for condition, (_, fast), (_, slow), k in decimation_pairs(
[(label, s) for label, _, s in records]): [(label, s) for label, _, s in records]):
try: try:
v = ch.decimate(dict(fast["cap"].axes())["total"], k, "fir") v = ch.decimate(fast["axes"]["total"], k, "fir")
except ValueError: except ValueError:
continue continue
hue = color[condition] hue = color[condition]
freqs, asd = binned_asd(*ch.welch_asd(v, fast["fs"] / k, freqs, asd = binned_asd(*ch.welch_asd(v, fast["fs"] / k,
nperseg=ch.segment_length(fast["fs"] / k))) nperseg=ch.segment_length(fast["fs"] / k)))
# Fractional again: the two cycle counts do not share a gain either. # Fractional again: the two cycle counts have their own gains too.
asd = asd / fast["field"] * 1e6 asd = asd / fast["field"] * 1e6
ax.loglog(freqs, asd, color=hue, linestyle="--", linewidth=1.5, ax.loglog(freqs, asd, color=hue, linestyle="--", linewidth=1.5,
label=f"{condition} cc{fast['cycle_count']} ÷{k}") label=f"{condition} cc{fast['cycle_count']} ÷{k}")
freqs, asd = binned_asd(*ch.welch_asd( freqs, asd = binned_asd(*ch.welch_asd(
dict(slow["cap"].axes())["total"], slow["fs"], slow["axes"]["total"], slow["fs"],
nperseg=ch.segment_length(slow["fs"]))) nperseg=ch.segment_length(slow["fs"])))
asd = asd / slow["field"] * 1e6 asd = asd / slow["field"] * 1e6
ax.loglog(freqs, asd, color=hue, linestyle="-", linewidth=1.5, ax.loglog(freqs, asd, color=hue, linestyle="-", linewidth=1.5,
@ -892,7 +851,7 @@ def _plot_corrections(ax, records, supplies, color, marker, cal=None):
ax.set_yticks([-r for r in range(len(rows))], [n for _, n in rows]) ax.set_yticks([-r for r in range(len(rows))], [n for _, n in rows])
ax.set_ylim(-len(rows) + 0.5, 0.85) ax.set_ylim(-len(rows) + 0.5, 0.85)
ax.margins(x=0.24) ax.margins(x=0.24)
title(ax, "Total field under each candidate correction") ch.title(ax, "Total field under each candidate correction")
ax.set_xlabel("|B| (nT)") ax.set_xlabel("|B| (nT)")
ax.annotate("a correction that is right collapses the spread", ax.annotate("a correction that is right collapses the spread",
xy=(0.02, 0.06), xycoords="axes fraction", xy=(0.02, 0.06), xycoords="axes fraction",
@ -932,7 +891,7 @@ def main():
"ratiometric correction; repeatable") "ratiometric correction; repeatable")
ap.add_argument("--calibration", default=None, metavar="JSON", ap.add_argument("--calibration", default=None, metavar="JSON",
help="reference the rate correction to a measured " help="reference the rate correction to a measured "
"calibration instead of rm3100's nominal timing model") "calibration in place of rm3100's nominal model")
ap.add_argument("-o", "--output", default=None, metavar="PNG", ap.add_argument("-o", "--output", default=None, metavar="PNG",
help="write a comparison figure here") help="write a comparison figure here")
args = ap.parse_args() args = ap.parse_args()
@ -963,15 +922,14 @@ def main():
file=sys.stderr) file=sys.stderr)
# One band for every capture, so no run is scored over more bandwidth than # One band for every capture, so no run is scored over more bandwidth than
# another. Without this a 299 Hz capture is judged over 3-135 Hz and a # another; otherwise the difference in bandwidth reads as a noise difference.
# 73 Hz one over 3-33, and the difference reads as a noise difference.
slowest = min(comparable_rates([(l, cap) for l, _, cap, _ in loaded])) slowest = min(comparable_rates([(l, cap) for l, _, cap, _ in loaded]))
band = args.band or ch.band_for(slowest) band = args.band or ch.band_for(slowest)
if band[1] > ch.BAND_NYQUIST_FRACTION * slowest / 2: if band[1] > ch.BAND_NYQUIST_FRACTION * slowest / 2:
print(f"warning: --band reaches {band[1]:g} Hz, past " print(f"warning: --band reaches {band[1]:g} Hz, past "
f"{ch.BAND_NYQUIST_FRACTION * slowest / 2:.2f} Hz where the " f"{ch.BAND_NYQUIST_FRACTION * slowest / 2:.2f} Hz where the "
f"slowest path compared -- {slowest:.2f} Hz, decimation " f"slowest path compared ({slowest:.2f} Hz, decimation "
f"included -- rolls off", file=sys.stderr) f"included) rolls off", file=sys.stderr)
records = [(label, path, summarise(cap, band)) for label, path, cap, _ in loaded] records = [(label, path, summarise(cap, band)) for label, path, cap, _ in loaded]

12
diagnose-comms.py
View file

@ -2,7 +2,7 @@
"""Hardware diagnostics for the RM3100 / CH347 link. """Hardware diagnostics for the RM3100 / CH347 link.
Walks the signal chain from USB inwards and stops at the first thing that is Walks the signal chain from USB inwards and stops at the first thing that is
broken, so a failure points at a specific wire rather than "it doesn't work": broken, so a failure points at a specific wire:
adapter -> I2C bus -> sensor identity -> registers -> analog section adapter -> I2C bus -> sensor identity -> registers -> analog section
@ -23,7 +23,7 @@ BUS_KHZ = ch347.DEFAULT_SPEED_KHZ
def check(label, ok, detail=""): def check(label, ok, detail=""):
print(f" [{'OK ' if ok else 'FAIL'}] {label}" + (f" -- {detail}" if detail else "")) print(f" [{'OK ' if ok else 'FAIL'}] {label}" + (f": {detail}" if detail else ""))
return ok return ok
@ -76,7 +76,7 @@ def main():
print("\n-> Registers are unreliable. Try a lower bus speed or shorter wires.") print("\n-> Registers are unreliable. Try a lower bus speed or shorter wires.")
return 1 return 1
print("\n5. Analog section (BIST -- drives the sensor coils)") print("\n5. Analog section (BIST, which drives the sensor coils)")
sensor.write_reg(rm3100.REG_BIST, [BIST_RUN]) sensor.write_reg(rm3100.REG_BIST, [BIST_RUN])
sensor.write_reg(rm3100.REG_POLL, [rm3100.POLL_ALL_AXES]) sensor.write_reg(rm3100.REG_POLL, [rm3100.POLL_ALL_AXES])
time.sleep(0.5) time.sleep(0.5)
@ -102,8 +102,8 @@ def main():
print("\n6. Live measurement") print("\n6. Live measurement")
sensor.configure() sensor.configure()
# BIST zeroes the measurement registers and leaves DRDY set. With # BIST zeroes the measurement registers and leaves DRDY set. With
# DRC1=1 only a results read clears it, so discard one -- otherwise # DRC1=1 only a results read clears it, so discard one; wait_for_data()
# wait_for_data() returns instantly on the stale flag and reports zeros. # would otherwise return instantly on the stale flag and report zeros.
sensor.read_raw() sensor.read_raw()
sensor.set_rate(0x96) sensor.set_rate(0x96)
sensor.start_cmm() sensor.start_cmm()
@ -129,7 +129,7 @@ def main():
finally: finally:
sensor.stop_cmm() sensor.stop_cmm()
print("\nAll checks passed -- the link is fully working.") print("\nAll checks passed, the link is fully working.")
return 0 return 0

381
logger.py
View file

@ -7,15 +7,13 @@ Run ./setup.sh first to install the udev rule and create the virtualenv, then:
./.venv/bin/python logger.py --scan-only ./.venv/bin/python logger.py --scan-only
The chip samples on its own internal schedule (manual sections 5.7.2, 5.8.2), so The chip samples on its own internal schedule (manual sections 5.7.2, 5.8.2), so
`sample_index` is a grid coordinate, not a count of reads. A late read returns the `sample_index` is a grid coordinate. A late read returns the *newer* measurement,
*newer* measurement rather than a delayed one, so an unnoticed miss would silently so every miss is detected and written as an explicit placeholder row, which keeps
compress the time axis. Misses are therefore detected and written as explicit the index contiguous and the gap visible. The run continues and reports the total
placeholder rows, keeping the index contiguous and the gap visible; the run at the end.
continues and reports the total at the end.
Only irreducible facts are written: the sample count, the host clock, and the raw Only irreducible facts are written: the sample count, the host clock, and the raw
counts. Chip time, tesla and magnitude are all reconstructed on load from the counts. Chip time, tesla and magnitude are reconstructed on load by capture.py.
header -- see capture.py.
""" """
import argparse import argparse
@ -34,35 +32,30 @@ import rm3100
CSV_FIELDS = ["sample_index", "system_time_unix", "x_raw", "y_raw", "z_raw", CSV_FIELDS = ["sample_index", "system_time_unix", "x_raw", "y_raw", "z_raw",
"warning"] "warning"]
# Mains fundamental. Anything sampled below twice this folds the line onto # Mains fundamental. Below twice this the line folds onto signal permanently,
# signal irreversibly, which is the one error post-processing cannot undo. # which is the one error post-processing cannot undo.
MAINS_HZ = 60.0 MAINS_HZ = 60.0
CONSOLE_REFRESH_S = 0.05 CONSOLE_REFRESH_S = 0.05
FLUSH_INTERVAL_S = 0.5 FLUSH_INTERVAL_S = 0.5
# How long to let the writer finish its backlog once sampling has stopped. Far # How long to let the writer finish its backlog once sampling has stopped. Far
# longer than a healthy writer needs; exceeding it means rows are lost, which is # longer than a healthy writer needs, so reaching it means rows are lost and the
# reported rather than waited out indefinitely. # run says so.
WRITER_DRAIN_TIMEOUT_S = 30.0 WRITER_DRAIN_TIMEOUT_S = 30.0
# The sampling loop releases the GIL inside each USB transfer, then has to take # The sampling loop releases the GIL inside each USB transfer and has to take it
# it back. Python's default 5 ms switch interval means the writer thread can # back, so Python's default 5 ms switch interval becomes the jitter floor: the
# hold it for that long, stalling the sampler and inflating the DRDY bracket -- # writer can hold the GIL that long, stalling the sampler and inflating the DRDY
# measured at ~10 ms against a 6.67 ms budget at 150 Hz, while the same loop # bracket past its budget. Handing off 10x more often costs little throughput,
# without a writer thread peaked at 3.1 ms. Handing off 10x more often costs # the writer being I/O bound.
# negligible throughput (the writer is not CPU-bound) and keeps the stall well
# inside the bracket budget.
GIL_SWITCH_INTERVAL_S = 0.0005 GIL_SWITCH_INTERVAL_S = 0.0005
# Thread priority. The sampling loop runs on the main thread and is the only # Thread priority. The sampling loop runs on the main thread and is the only
# latency-sensitive part; the writer merely formats a few fields per sample. # latency-sensitive part. Lowering the writer needs no privilege and is always
# # done: the *relative* priority is what matters, and on Linux nice is per-thread.
# Lowering the writer needs no privilege and is always done: only the *relative* # Raising the sampler needs CAP_SYS_NICE, so it is opt-in via --high-priority,
# priority matters, and on Linux nice is per-thread, so it does not touch the # which fails loudly when the privilege is missing.
# sampler. Raising the sampler needs CAP_SYS_NICE and is opt-in via
# --high-priority, which fails loudly rather than degrading silently -- asking
# for it and not getting it is worth knowing about.
SAMPLER_NICE = -10 SAMPLER_NICE = -10
WRITER_NICE = 10 WRITER_NICE = 10
@ -70,10 +63,8 @@ WRITER_NICE = 10
def restore_ownership(path): def restore_ownership(path):
"""Hand a file created under sudo back to the invoking user. """Hand a file created under sudo back to the invoking user.
--high-priority needs root, and anything root writes stays root-owned -- --high-priority needs root and anything root writes stays root-owned, so
which earlier in this project produced captures the normal user could not the ownership is handed back here. Privilege buys scheduling and nothing else.
rewrite. Undo that here so privilege is needed for scheduling and nothing
else leaks from it.
""" """
if os.geteuid() != 0: if os.geteuid() != 0:
return return
@ -91,9 +82,8 @@ def set_thread_nice(value):
"""Renice the calling thread. Returns the nice actually in effect, or None. """Renice the calling thread. Returns the nice actually in effect, or None.
Linux threads are tasks, so PRIO_PROCESS with a native thread id applies to Linux threads are tasks, so PRIO_PROCESS with a native thread id applies to
just this thread. Elsewhere this may be a no-op or affect the process, which just this thread. Elsewhere it may be a no-op or reach the whole process.
is why failure is tolerated rather than fatal -- priority is an optimisation, Failure is tolerated, priority being an optimisation.
not a correctness requirement.
""" """
try: try:
tid = threading.get_native_id() tid = threading.get_native_id()
@ -102,59 +92,19 @@ def set_thread_nice(value):
except (AttributeError, OSError): except (AttributeError, OSError):
return None return None
# Miss detection uses the DRDY bracket -- the span between the last poll showing # Bounded, so a stalled writer degrades predictably on a long run. Far above the
# DRDY clear and the poll showing it set -- rather than the read-to-read # depth a healthy writer reaches, so filling it means the writer is wedged. The
# interval. # run then ends, which keeps sample_index contiguous over everything written.
#
# The bracket is exact where the interval is only a heuristic. If DRDY reads
# clear at t_c then every earlier measurement has already been read, since a
# results read is what clears it. Measurements complete one period apart, so a
# bracket narrower than one period can contain at most one completion, and DRDY
# going high proves it contained at least one. Exactly one, whatever the host
# was doing beforehand.
#
# The read-to-read interval cannot make that claim: measured on this rig it
# reaches 37 ms against a 28.8 ms period -- a 29% overshoot from host stalls
# alone -- while the bracket stays under 10 ms. Thresholding the interval
# therefore flags healthy captures as lossy.
#
# The bracket is compared against the period measured by calibrate_period(), not
# the table value: the latter is 6-9% out on this unit, which is enough to
# miscount grid points inside a bracket and slip the sample index.
# Bounded so a stalled writer degrades predictably instead of exhausting memory
# on a long run. Far above the depth a healthy writer ever reaches, so filling it
# means the writer is wedged rather than merely behind -- and the run then ends,
# because a row that never reaches the file would leave a hole in sample_index
# and capture.py rejects a non-contiguous index outright. Stopping keeps
# everything already written valid.
QUEUE_MAX = 200_000 QUEUE_MAX = 200_000
_SENTINEL = object() _SENTINEL = object()
# Rows carry flags in a `warning` column, space separated, empty when fine. # Rows carry flags in a `warning` column, space separated, empty when fine. See
# Keeping them out of the numeric columns means x/y/z stay parseable as numbers # the capture format in the README; the two are independent, so a row can carry
# (blank for a row with no data) and the column generalises to future flags. # either or both.
# WARN_MISSED = "MISSED" # placeholder: the measurement was never read
# MISSED this row is a placeholder -- the measurement was never read, so it WARN_AMBIGUOUS = "AMBIGUOUS" # gap of uncertain length ends here
# has no data. The row exists to keep sample_index contiguous, so it
# remains a valid chip-time grid coordinate and the gap is explicit
# rather than silently compressing the timeline.
# AMBIGUOUS the gap ending at this row was of uncertain length, so the index
# may have slipped from here on.
#
# The two are independent. A gap measuring 1.35 periods rounds to one, so no
# placeholder is written -- yet it sits far enough from an integer to distrust,
# and that case carries AMBIGUOUS on the *real* sample terminating the gap,
# which keeps its data. A placeholder inside an uncertain gap carries both.
#
# The flag is a confidence measure, not a claim that 1.35 and 1.65 are equally
# likely: 1.35 probably is one period and 1.65 probably two. Deciding which
# needs neighbouring timestamps and assumptions, and can still be wrong, which
# is exactly why the rows are flagged rather than silently resolved.
WARN_MISSED = "MISSED"
WARN_AMBIGUOUS = "AMBIGUOUS"
# How near a half-period the rounding may fall before the count is a coin toss. # How near a half-period the rounding may fall before the count is a coin toss.
AMBIGUITY_MARGIN = 0.25 AMBIGUITY_MARGIN = 0.25
@ -176,7 +126,7 @@ def parse_args():
p.add_argument("--tmrc", type=lambda s: int(s, 0), default=None, p.add_argument("--tmrc", type=lambda s: int(s, 0), default=None,
help="continuous-mode rate register, 0x92 (fastest) to 0x9D " help="continuous-mode rate register, 0x92 (fastest) to 0x9D "
"(slowest). Default is the fastest, which lets the cycle " "(slowest). Default is the fastest, which lets the cycle "
"count set the rate -- give one only to sample slower " "count set the rate; give one only to sample slower "
"than the cycle count allows") "than the cycle count allows")
p.add_argument("--duration", type=float, default=0.0, p.add_argument("--duration", type=float, default=0.0,
help="seconds to log, or 0 to run until Ctrl-C (default: %(default)s)") help="seconds to log, or 0 to run until Ctrl-C (default: %(default)s)")
@ -225,9 +175,9 @@ def print_plan(cfg, bus_speed, requested_rate, sampler_nice=None):
f"{rm3100.AXIS_OVERHEAD_S * 1e6:.1f} us))") f"{rm3100.AXIS_OVERHEAD_S * 1e6:.1f} us))")
print(f" sensor ceiling {1 / rm3100.sample_period(cfg.cycle_count):.2f} Hz") print(f" sensor ceiling {1 / rm3100.sample_period(cfg.cycle_count):.2f} Hz")
print(f" TMRC 0x{cfg.tmrc:02x} = " print(f" TMRC 0x{cfg.tmrc:02x} = "
f"{rm3100.TMRC_RATES[cfg.tmrc]:g} Hz -- {cfg.governed_by} governs") f"{rm3100.TMRC_RATES[cfg.tmrc]:g} Hz, {cfg.governed_by} governs")
print(f" predicted rate {cfg.predicted_hz:.2f} Hz " print(f" predicted rate {cfg.predicted_hz:.2f} Hz "
f"+/-{rm3100.RATE_TOLERANCE:.0%} (oscillator tolerance; measured below)") f"+/-{rm3100.RATE_TOLERANCE:.0%} (nominal-model spread; measured below)")
print(f" resolution {lsb:.2f} nT/LSB dither {dither:.2f} LSB " print(f" resolution {lsb:.2f} nT/LSB dither {dither:.2f} LSB "
"at spec noise") "at spec noise")
print(f" duty {cfg.duty:.1%} integration / period") print(f" duty {cfg.duty:.1%} integration / period")
@ -247,40 +197,46 @@ def print_plan(cfg, bus_speed, requested_rate, sampler_nice=None):
# wrong and what would fix it, then run what was requested. # wrong and what would fix it, then run what was requested.
warnings = [] warnings = []
# Below the recommended cycle count the LSB grows faster than the sensor's # What a low cycle count costs is duty; see rm3100.MIN_CYCLE_COUNT. Dither
# own noise, so the quantiser stops being dithered and averaging stalls. # is the more commonly cited worry, so it is reported alongside.
# The manual's hard floor is 30 (section 5.1), but the margin is already
# slim at 50, so anything under that is worth saying out loud.
if cfg.cycle_count < rm3100.RECOMMENDED_MIN_CYCLE_COUNT: if cfg.cycle_count < rm3100.RECOMMENDED_MIN_CYCLE_COUNT:
reference_duty = (rm3100.integration_time(rm3100.DEFAULT_CYCLE_COUNT)
/ rm3100.sample_period(rm3100.DEFAULT_CYCLE_COUNT))
penalty = (reference_duty / cfg.duty) ** 0.5 - 1.0
warnings.append( warnings.append(
f"cycle count {cfg.cycle_count} is below the recommended " f"cycle count {cfg.cycle_count} is below the recommended "
f"{rm3100.RECOMMENDED_MIN_CYCLE_COUNT}: dither is {dither:.2f} LSB " f"{rm3100.RECOMMENDED_MIN_CYCLE_COUNT}: the per-axis overhead holds "
f"at spec noise ({lsb:.1f} nT/LSB) against the ~0.2 where averaging " f"duty to {cfg.duty:.0%} against {reference_duty:.0%} at cycle "
"stops recovering sub-LSB resolution. characterize.py prints sd/LSB " f"count {rm3100.DEFAULT_CYCLE_COUNT}, which costs {penalty:.0%} of "
f"post-filter noise floor. Dither is {dither:.2f} LSB at spec noise "
f"({lsb:.1f} nT/LSB); characterize.py prints the measured sd/LSB "
"per axis, which settles it from the first capture.") "per axis, which settles it from the first capture.")
# The measured worst configuration mistake -- TMRC slower than the # The worst configuration mistake: TMRC slower than the cycle-count ceiling
# cycle-count ceiling leaves the sensor idle, and idle time buys nothing. # leaves the sensor idle, and idle time buys nothing. Idle is the shortfall
# Idle is the shortfall against the ceiling, which is what TMRC costs; it is # against the ceiling; the per-axis overhead is active time too.
# not 1 - duty, since the fixed per-axis overhead is active time too.
idle = 1.0 - cfg.predicted_hz * rm3100.sample_period(cfg.cycle_count) idle = 1.0 - cfg.predicted_hz * rm3100.sample_period(cfg.cycle_count)
if cfg.governed_by == "TMRC" and idle > 0.2: if cfg.governed_by == "TMRC" and idle > 0.2:
# Against the duty the *same rate* would reach on the cycle count alone,
# which is not 100%: the per-axis overhead is a fixed cost per sample, so
# even a free-running sensor gives some of it back at high rates.
best_duty = 1.0 - rm3100.AXES * rm3100.AXIS_OVERHEAD_S * cfg.predicted_hz
warnings.append( warnings.append(
f"TMRC 0x{cfg.tmrc:02x} governs and leaves the sensor idle " f"TMRC 0x{cfg.tmrc:02x} governs and leaves the sensor idle "
f"{idle:.0%} of each period, holding duty to {cfg.duty:.0%}. " f"{idle:.0%} of each period, holding duty to {cfg.duty:.0%} against "
"Measured cost at 23% duty was 1.43x the noise ASD. Raising the " f"the {best_duty:.0%} this rate reaches on the cycle count alone. "
"cycle count instead reaches the same rate at ~100% duty.") f"Post-filter noise goes as 1/sqrt(duty), so that is "
f"{(best_duty / cfg.duty) ** 0.5:.2f}x the noise floor for nothing.")
# A rate that came back different from the one asked for is easy to miss in # Every figure above is derived from the rate, so a rate that came back
# a note, and every figure above is derived from the rate. # different from the one asked for is worth saying out loud.
if (requested_rate is not None and abs(cfg.predicted_hz / requested_rate - 1) if (requested_rate is not None and abs(cfg.predicted_hz / requested_rate - 1)
> rm3100.RATE_TOLERANCE): > rm3100.RATE_TOLERANCE):
warnings.append( warnings.append(
f"{requested_rate:g} Hz was requested but this configuration runs " f"{requested_rate:g} Hz was requested but this configuration runs "
f"at {cfg.predicted_hz:.2f} Hz.") f"at {cfg.predicted_hz:.2f} Hz.")
# Mains and its harmonics fold onto signal below 2x mains, and no amount of # Below 2x mains the line folds onto signal, and no filter undoes an alias.
# post-filtering undoes an alias.
if cfg.predicted_hz < 2 * MAINS_HZ: if cfg.predicted_hz < 2 * MAINS_HZ:
alias = abs(MAINS_HZ - round(MAINS_HZ / cfg.predicted_hz) * cfg.predicted_hz) alias = abs(MAINS_HZ - round(MAINS_HZ / cfg.predicted_hz) * cfg.predicted_hz)
warnings.append( warnings.append(
@ -289,8 +245,6 @@ def print_plan(cfg, bus_speed, requested_rate, sampler_nice=None):
"afterwards. Sampling faster and decimating gives the same noise " "afterwards. Sampling faster and decimating gives the same noise "
"floor with the mains line still visible.") "floor with the mains line still visible.")
# Table 3-1 stops at 400, so the noise figure printed above is extrapolation
# past that point rather than a specification.
if cfg.cycle_count > rm3100.MAX_SPEC_CYCLE_COUNT: if cfg.cycle_count > rm3100.MAX_SPEC_CYCLE_COUNT:
warnings.append( warnings.append(
f"cycle count {cfg.cycle_count:,} is past the {rm3100.MAX_SPEC_CYCLE_COUNT} " f"cycle count {cfg.cycle_count:,} is past the {rm3100.MAX_SPEC_CYCLE_COUNT} "
@ -342,7 +296,7 @@ def start_sensor(sensor, cycle_count, tmrc):
"""Identify and configure the sensor, printing each step. Returns REVID.""" """Identify and configure the sensor, printing each step. Returns REVID."""
revid = sensor.revid() revid = sensor.revid()
if revid == rm3100.EXPECTED_REVID: if revid == rm3100.EXPECTED_REVID:
print(f"REVID 0x{revid:02x} -- RM3100 confirmed") print(f"REVID 0x{revid:02x}, RM3100 confirmed")
else: else:
print(f"WARNING: REVID 0x{revid:02x}, expected " print(f"WARNING: REVID 0x{revid:02x}, expected "
f"0x{rm3100.EXPECTED_REVID:02x}", file=sys.stderr) f"0x{rm3100.EXPECTED_REVID:02x}", file=sys.stderr)
@ -354,7 +308,7 @@ def start_sensor(sensor, cycle_count, tmrc):
f"{(cycle_count,) * 3}", file=sys.stderr) f"{(cycle_count,) * 3}", file=sys.stderr)
sys.exit(1) sys.exit(1)
lsb_nt = rm3100.tesla_per_count(cycle_count) * rm3100.NT_PER_TESLA lsb_nt = rm3100.tesla_per_count(cycle_count) * rm3100.NT_PER_TESLA
print(f"Cycle counts set to {readback} -- 1 LSB = {lsb_nt:.2f} nT") print(f"Cycle counts set to {readback}, 1 LSB = {lsb_nt:.2f} nT")
# Raises if DRC1 did not take; exactly-once sampling depends on it. # Raises if DRC1 did not take; exactly-once sampling depends on it.
sensor.configure() sensor.configure()
@ -388,11 +342,10 @@ def writer_thread(q, path, meta, stats):
break break
index, system_time, cx, cy, cz, warning = item index, system_time, cx, cy, cz, warning = item
# 6 decimal places is microsecond resolution on an epoch value, # 6 decimals is microsecond resolution on an epoch value, far finer
# far finer than the ~1 ms bracket uncertainty on each timestamp. # than the ~1 ms bracket uncertainty. Placeholders carry zeros rather
# Placeholder rows carry zeros rather than blanks: the warning # than blanks: the warning column already says the row has no data,
# column already says the row has no data, so keeping x/y/z strictly # so x/y/z stay strictly integer and trivial to parse.
# integer makes the file trivial to parse.
out.writerow([index, f"{system_time:.6f}", out.writerow([index, f"{system_time:.6f}",
0 if cx is None else cx, 0 if cx is None else cx,
0 if cy is None else cy, 0 if cy is None else cy,
@ -405,7 +358,7 @@ def writer_thread(q, path, meta, stats):
last_flush = now last_flush = now
if now - last_print >= CONSOLE_REFRESH_S and cx is not None: if now - last_print >= CONSOLE_REFRESH_S and cx is not None:
# Placeholders carry no field values, so the display holds the # Placeholders carry no field values, so the display holds the
# last real reading rather than blanking or showing nonsense. # last real reading.
last_print = now last_print = now
x, y, z = cx * lsb_nt, cy * lsb_nt, cz * lsb_nt x, y, z = cx * lsb_nt, cy * lsb_nt, cz * lsb_nt
print(f"\r{stats['rows']:,} samples ({stats['missed']} missed)" print(f"\r{stats['rows']:,} samples ({stats['missed']} missed)"
@ -418,14 +371,14 @@ def writer_thread(q, path, meta, stats):
def i2c_bus_time(bus_speed_khz): def i2c_bus_time(bus_speed_khz):
"""Seconds of I2C traffic the bus carries per sample. """Seconds of I2C traffic the bus carries per sample.
A combined register read of n payload bytes clocks n+3 bytes -- write A combined register read of n payload bytes clocks n+3 bytes (write address,
address, register, read address, payload -- at 9 bit-times each (8 data plus register, read address, payload) at 9 bit-times each, 8 data plus ACK, and
ACK), and spends roughly a bit-time on each START and STOP. One sample costs spends roughly a bit-time on each START and STOP. One sample costs a STATUS
a STATUS poll (1 payload byte) plus a results read (9). poll of 1 payload byte plus a results read of 9.
This is the irreducible traffic, not bus occupancy: the loop polls This is the irreducible traffic, and the figure worth reading against the
continuously while waiting for DRDY, so raw occupancy approaches 100% and sample period. The loop polls continuously while waiting for DRDY, so raw
says nothing useful. What matters is this against the sample period. bus occupancy approaches 100%.
""" """
bits = 0 bits = 0
for payload in (1, 9): for payload in (1, 9):
@ -436,34 +389,29 @@ def i2c_bus_time(bus_speed_khz):
def calibrate_period(sensor, dt_nominal, seconds, attempts=3): def calibrate_period(sensor, dt_nominal, seconds, attempts=3):
"""Measure the true measurement period from a clean stretch of samples. """Measure the true measurement period from a clean stretch of samples.
Miss detection needs the real period, not the table value: the RC oscillator Miss detection needs the real period. The table value is a nominal model,
runs up to 7% off (manual section 5.2.1) and this unit is ~6% slow, which is out by up to ~6% on this unit, which is enough to miscount grid points
enough to miscount grid points inside a bracket and slip the sample index. inside a bracket and slip the sample index.
Least-squares slope of read time against index. Over one second at 35 Hz Least-squares slope of read time against index, which over one second pins
(~35 points, ~0.5 ms jitter) that pins the period to roughly 0.03%, against the period to roughly 0.03%, far finer than the resolution needed to tell k
the 6-9% error of the nominal value -- and 0.03% is far finer than the grid points from k+1.
resolution needed to tell k grid points from k+1.
Requires a stretch with no miss, since an interval spanning a miss is a Requires a stretch with no miss, since an interval spanning one is a multiple
multiple of the period, not one period. Returns None if no clean stretch is of the period. Returns None when no clean stretch is available, which is
available, which is itself the answer: the rate is not sustainable. itself the answer: the rate is not sustainable.
Also returns the mean host cost of one sample's traffic -- a DRDY poll plus Also returns the mean host cost of one sample's traffic, a DRDY poll plus a
a results read -- timed here at no extra cost, since those are the very results read, timed here for free from the transactions being performed.
transactions being performed.
""" """
poll_ready, read_raw = sensor.poll_ready, sensor.read_raw poll_ready, read_raw = sensor.poll_ready, sensor.read_raw
monotonic = time.monotonic monotonic = time.monotonic
# Bootstrap the period from observation rather than trusting the TMRC table. # Bootstrap the period from observation rather than the TMRC table. When the
# When the cycle count governs instead of TMRC -- which it does whenever the # cycle count governs, the table value is out by a large factor: at cc=400
# cycle-count ceiling falls below the requested rate (manual section 5.2.1) # with TMRC 0x92 it says 1.67 ms against a real 13.5 ms, and a threshold
# -- the table value is not merely 7% out but wrong by a large factor. At # built on it rejects every interval. The median of a handful of raw
# cc=400 with TMRC 0x92 the table says 1.67 ms and the truth is 13.5 ms, so # intervals is robust to the occasional stall and seeds the real fit.
# a threshold built on it rejects every interval and calibration can never
# start. A handful of raw intervals settles it: the median is robust to the
# occasional stall, and only needs to be close enough to seed the real fit.
probe = [] probe = []
clear = monotonic() clear = monotonic()
deadline = clear + max(seconds, 0.25) deadline = clear + max(seconds, 0.25)
@ -496,9 +444,8 @@ def calibrate_period(sensor, dt_nominal, seconds, attempts=3):
if ready - clear >= dt_nominal: # conservative: nominal is short if ready - clear >= dt_nominal: # conservative: nominal is short
clean = False clean = False
break break
# Timed from here, not from `ready`: `ready` is the instant the # Timed from here. `ready` is the instant the STATUS reply landed
# STATUS reply landed on the wire, so measuring against it would # on the wire, so it carries the tail of the poll transaction.
# charge the tail of the poll transaction to the read as well.
before = monotonic() before = monotonic()
read_raw() read_raw()
read_cost += monotonic() - before read_cost += monotonic() - before
@ -519,23 +466,27 @@ def calibrate_period(sensor, dt_nominal, seconds, attempts=3):
def sample_loop(sensor, q, duration, dt_nominal, stats): def sample_loop(sensor, q, duration, dt_nominal, stats):
"""Read every measurement, recording placeholders for any that are lost. """Read every measurement, recording placeholders for any that are lost.
The timestamp recorded per sample is the midpoint of the bracket between the Miss detection uses the **DRDY bracket**: the span between the last poll
last poll showing DRDY clear and the poll showing it set: the measurement showing DRDY clear and the poll showing it set. The bracket is exact. A
completed somewhere in that window, and the midpoint is the best estimate results read is what clears DRDY, so everything earlier has been read;
available without a hardware DRDY line. measurements complete one period apart, so a bracket narrower than a period
holds at most one completion, and DRDY going high proves it held at least
one. Host stalls push the read-to-read interval well past a period, and this
is unaffected by them.
Ends the run rather than dropping a row if the queue fills: sample_index has The timestamp recorded per sample is the midpoint of that bracket, the best
to stay contiguous to remain a chip-time grid coordinate, so a truncated estimate available without a hardware DRDY line.
capture is worth more than a longer one with a hole in it.
Ends the run if the queue fills, which keeps sample_index contiguous and so
keeps every row written a valid chip-time grid coordinate.
""" """
read_raw = sensor.read_raw read_raw = sensor.read_raw
poll_ready = sensor.poll_ready poll_ready = sensor.poll_ready
monotonic = time.monotonic monotonic = time.monotonic
put = q.put_nowait put = q.put_nowait
# DRDY may already be set from whatever ran before -- BIST in particular # DRDY may already be set from whatever ran before; BIST zeroes the result
# zeroes the result registers and leaves it high. Discard one reading so # registers and leaves it high. Discard one reading so sample 0 is fresh.
# sample 0 is a genuinely fresh measurement.
if poll_ready()[0]: if poll_ready()[0]:
read_raw() read_raw()
@ -546,7 +497,7 @@ def sample_loop(sensor, q, duration, dt_nominal, stats):
# dt_nominal here is the *calibrated* period from calibrate_period(), not # dt_nominal here is the *calibrated* period from calibrate_period(), not
# the table value. Keep refining it from clean intervals so the estimate # the table value. Keep refining it from clean intervals so the estimate
# follows the oscillator's thermal drift (~2500 ppm measured over 13 h). # follows the oscillator's thermal drift.
dt_est = dt_nominal dt_est = dt_nominal
previous_mono = previous_wall = None previous_mono = previous_wall = None
DT_SMOOTHING = 0.02 DT_SMOOTHING = 0.02
@ -555,7 +506,7 @@ def sample_loop(sensor, q, duration, dt_nominal, stats):
if monotonic() >= deadline: if monotonic() >= deadline:
return return
# The stamps come back with the STATUS reply itself, taken the instant # The stamps come back with the STATUS reply itself, taken the instant
# it landed rather than several frames later up here. # it landed.
ready, ready_mono, ready_wall = poll_ready() ready, ready_mono, ready_wall = poll_ready()
if not ready: if not ready:
clear_mono, clear_wall = ready_mono, ready_wall clear_mono, clear_wall = ready_mono, ready_wall
@ -566,34 +517,20 @@ def sample_loop(sensor, q, duration, dt_nominal, stats):
if bracket > stats["max_bracket"]: if bracket > stats["max_bracket"]:
stats["max_bracket"] = bracket stats["max_bracket"] = bracket
# Counting lost measurements in two stages, because the two questions # Two stages, because the two questions have different best answers.
# have different best answers. # "Did we lose any?" is settled rigorously by the bracket, above. "How
# # many?" is better answered by the interval since the previous accepted
# "Did we lose any?" is settled by the bracket, and rigorously: a # sample: each accepted read sits on a grid point, so that interval is
# bracket shorter than one period cannot contain two completions, so # very close to an exact multiple of the period and rounding recovers
# nothing was lost regardless of timing precision. # the multiple. The bracket reaches back only to the last poll that saw
# # DRDY clear, which loses track of where the grid sits.
# "How many?" is better answered by the interval since the previous
# accepted sample. Each accepted read sits on a grid point, so that
# interval is very close to an exact multiple of the period, and
# rounding recovers the multiple. The bracket alone cannot -- it only
# spans back to the last poll that saw DRDY clear, which discards where
# the grid actually is.
#
# Precision: the period is calibrated to ~0.07% and each completion is
# located to about half a poll interval (~0.2 ms, under 1% of a period).
# Over a gap of k periods the total error is well under a tenth of a
# period for any small k, so the rounding is unambiguous -- which is
# the case that matters, since a long gap means the run is unusable
# anyway.
lost = 0 lost = 0
uncertain = False uncertain = False
if bracket >= dt_est and previous_mono is not None: if bracket >= dt_est and previous_mono is not None:
periods = (ready_mono - previous_mono) / dt_est periods = (ready_mono - previous_mono) / dt_est
lost = max(0, round(periods) - 1) lost = max(0, round(periods) - 1)
# Uncertainty is a property of the gap, not of the placeholders: # Uncertainty is a property of the gap: one rounding to zero losses
# a gap rounding to zero losses can still be a coin toss, and then # can still be a coin toss, with no placeholder to carry the flag.
# there is no placeholder to carry the flag.
uncertain = abs(periods - round(periods)) > AMBIGUITY_MARGIN uncertain = abs(periods - round(periods)) > AMBIGUITY_MARGIN
if uncertain: if uncertain:
stats["ambiguous"] += 1 stats["ambiguous"] += 1
@ -603,11 +540,9 @@ def sample_loop(sensor, q, duration, dt_nominal, stats):
dt_est += (ready_mono - previous_mono - dt_est) * DT_SMOOTHING dt_est += (ready_mono - previous_mono - dt_est) * DT_SMOOTHING
if lost: if lost:
stats["missed"] += lost stats["missed"] += lost
# Placeholders sit on the grid -- one period after the previous # Placeholders sit on the grid, one period apart from the previous
# accepted sample, and so on -- because that is where the lost # accepted sample, which is where the lost measurements completed
# measurements actually completed. Spreading them across the # and what keeps the rate fit straight.
# bracket instead would bunch them at the end of the gap and skew
# the rate fit.
flags = (f"{WARN_MISSED} {WARN_AMBIGUOUS}" if uncertain flags = (f"{WARN_MISSED} {WARN_AMBIGUOUS}" if uncertain
else WARN_MISSED) else WARN_MISSED)
for k in range(lost): for k in range(lost):
@ -648,7 +583,7 @@ def main():
"Re-run under sudo, using the venv interpreter by absolute " "Re-run under sudo, using the venv interpreter by absolute "
"path:\n" "path:\n"
f" sudo {sys.executable} {' '.join(sys.argv)}\n" f" sudo {sys.executable} {' '.join(sys.argv)}\n"
"Or drop the flag -- the writer thread already steps aside, " "Or drop the flag: the writer thread already steps aside, "
f"which is the half that needs no privilege.") f"which is the half that needs no privilege.")
else: else:
try: try:
@ -694,9 +629,8 @@ def main():
print() print()
nominal_rate = cfg.predicted_hz nominal_rate = cfg.predicted_hz
# Establish the real period before recording. Everything downstream -- # Miss detection, placeholder counting and the sample-index grid all
# miss detection, placeholder counting, the sample-index grid -- depends # depend on the real period, so measure it first.
# on it, and the table value is 6-9% out on this unit.
dt, host_cost = calibrate_period(sensor, 1.0 / nominal_rate, dt, host_cost = calibrate_period(sensor, 1.0 / nominal_rate,
args.calibrate) args.calibrate)
if dt is None: if dt is None:
@ -708,16 +642,15 @@ def main():
"misses cannot be\ncounted reliably and the sample index would " "misses cannot be\ncounted reliably and the sample index would "
"not track chip time.\n" "not track chip time.\n"
"Lower the rate (higher --tmrc) or raise --bus-speed.") "Lower the rate (higher --tmrc) or raise --bus-speed.")
# Compared against the predicted rate, not TMRC's nominal: when the # Against the predicted rate, which is what the configuration aimed
# cycle count governs, the TMRC figure is not what was aimed for and the # for once the cycle count governs.
# error against it is meaningless.
print(f"Calibrated period {dt * 1e3:.4f} ms = {1 / dt:.4f} Hz " print(f"Calibrated period {dt * 1e3:.4f} ms = {1 / dt:.4f} Hz "
f"({(1 / dt) / cfg.predicted_hz - 1:+.2%} vs predicted)") f"({(1 / dt) / cfg.predicted_hz - 1:+.2%} vs predicted)")
# Whether the host is the constraint is a measurement, not a given: it # Whether the host is the constraint is a measurement. It costs a
# costs a near-constant ~0.9 ms per sample regardless of configuration, # near-constant ~0.9 ms per sample, so it dominates at short periods
# so it dominates at short periods and disappears at long ones. # and disappears at long ones.
share = host_cost / dt share = host_cost / dt
margin = ("little margin -- expect misses under load" if share > 0.5 margin = ("little margin, expect misses under load" if share > 0.5
else "modest margin" if share > 0.25 else "modest margin" if share > 0.25
else "ample margin") else "ample margin")
print(f"Host cost {host_cost * 1e3:.3f} ms/sample " print(f"Host cost {host_cost * 1e3:.3f} ms/sample "
@ -725,9 +658,7 @@ def main():
meta = { meta = {
"rm3100_capture": 1, "rm3100_capture": 1,
# The rate this configuration is predicted to produce, whichever of # Whichever of the two ceilings governs.
# the two ceilings governs -- not TMRC's table value, which is not
# what was aimed for when the cycle count wins.
"nominal_rate_hz": nominal_rate, "nominal_rate_hz": nominal_rate,
"tmrc_nominal_hz": rm3100.TMRC_RATES[args.tmrc], "tmrc_nominal_hz": rm3100.TMRC_RATES[args.tmrc],
"tmrc": f"0x{args.tmrc:02x}", "tmrc": f"0x{args.tmrc:02x}",
@ -737,9 +668,8 @@ def main():
"i2c_address": f"0x{address:02x}", "i2c_address": f"0x{address:02x}",
"bus_speed_khz": args.bus_speed, "bus_speed_khz": args.bus_speed,
"revid": f"0x{revid:02x}", "revid": f"0x{revid:02x}",
# What the logger actually used for miss detection. Not recoverable # What miss detection actually used. capture.py fits the whole run,
# from the data, since capture.py fits the whole run rather than the # so this figure lives only here.
# first second.
"calibrated_period_s": repr(dt), "calibrated_period_s": repr(dt),
} }
if args.note: if args.note:
@ -747,7 +677,7 @@ def main():
meta["note"] = " ".join(args.note.split()) meta["note"] = " ".join(args.note.split())
path = args.output or datetime.now().strftime("rm3100_%Y%m%d_%H%M%S.csv") path = args.output or datetime.now().strftime("rm3100_%Y%m%d_%H%M%S.csv")
print(f"Logging to {path} -- Ctrl-C to stop\n") print(f"Logging to {path}, Ctrl-C to stop\n")
q = queue.Queue(maxsize=QUEUE_MAX) q = queue.Queue(maxsize=QUEUE_MAX)
stats = {"rows": 0, "missed": 0, "ambiguous": 0, "max_bracket": 0.0, stats = {"rows": 0, "missed": 0, "ambiguous": 0, "max_bracket": 0.0,
@ -772,54 +702,53 @@ def main():
print(f"WARNING: could not stop CMM: {exc}", file=sys.stderr) print(f"WARNING: could not stop CMM: {exc}", file=sys.stderr)
print(f"Wrote {stats['rows']} samples to {path}") print(f"Wrote {stats['rows']} samples to {path}")
# How close the run came to losing a measurement: the useful number for # How close the run came to losing a measurement: the number to watch
# judging whether a rate is sustainable before committing to a long run. # before committing to a long run. Quoted against the calibrated period,
# Quoted against the calibrated period, which is the threshold actually # which is the threshold enforced.
# enforced -- the nominal one is 6-9% out.
print(f"Worst DRDY bracket {stats['max_bracket'] * 1e3:.2f} ms of " print(f"Worst DRDY bracket {stats['max_bracket'] * 1e3:.2f} ms of "
f"{dt * 1e3:.2f} ms allowed " f"{dt * 1e3:.2f} ms allowed "
f"({stats['max_bracket'] / dt * 100:.0f}% of margin used)") f"({stats['max_bracket'] / dt * 100:.0f}% of margin used)")
# Every one of these means the capture is usable but compromised in a # Each of these leaves the capture usable but compromised in a way the
# way the file alone does not show, so each is said out loud and each # file alone conceals, so each is said out loud and each makes the exit
# makes the exit status non-zero. # status non-zero.
failed = False failed = False
if stats["truncated"]: if stats["truncated"]:
failed = True failed = True
print("\nWARNING: recording stopped early -- the writer could not " print("\nWARNING: recording stopped early. The writer could not "
"keep up and the queue\nfilled. Continuing would have left a " "keep up and the\nqueue filled, so the run ended to keep "
"hole in sample_index, which capture.py rejects,\nso the run " "sample_index contiguous.\nEverything written is valid.",
"ended instead. Everything written is valid.", file=sys.stderr) file=sys.stderr)
if not drained: if not drained:
failed = True failed = True
# The writer is a daemon thread, so it dies with the process and # The writer is a daemon thread, so anything still queued dies with
# anything still queued never reaches the file. Say so: a short tail # the process. Say so, since a short tail is invisible in the file.
# is not visible from the file itself.
print(f"\nWARNING: the writer did not finish within " print(f"\nWARNING: the writer did not finish within "
f"{WRITER_DRAIN_TIMEOUT_S:g} s, so the last {q.qsize():,} " f"{WRITER_DRAIN_TIMEOUT_S:g} s, so the last\n"
"queued row(s) were\nnot written. The file is valid, just " f"{q.qsize():,} queued row(s) were dropped. The file is valid "
"short.", file=sys.stderr) "up to where it stops.", file=sys.stderr)
if stats["ambiguous"]: if stats["ambiguous"]:
failed = True failed = True
print(f"\nWARNING: {stats['ambiguous']:,} gap(s) could not be " print(f"\nWARNING: {stats['ambiguous']:,} gap(s) could not be "
"counted confidently -- the interval fell near a half-period, " "counted confidently. The interval fell near a\nhalf-period, "
"so\nthe number of lost measurements is a guess and " "so the number of lost measurements is a guess and "
f"sample_index may have slipped.\nThose rows carry " "sample_index\nmay have slipped from there on. Those rows "
f"{WARN_AMBIGUOUS} in the warning column. Do not trust this " f"carry {WARN_AMBIGUOUS} in the\nwarning column. Re-record "
"capture's\ntime axis for spectral work.", file=sys.stderr) "before using this capture's time axis.",
file=sys.stderr)
if stats["missed"]: if stats["missed"]:
failed = True failed = True
pct = stats["missed"] / max(stats["rows"], 1) * 100 pct = stats["missed"] / max(stats["rows"], 1) * 100
print(f"\nWARNING: {stats['missed']:,} measurement(s) were lost " print(f"\nWARNING: {stats['missed']:,} measurement(s) were lost "
f"({pct:.3f}% of rows), flagged {WARN_MISSED}.\n" f"({pct:.3f}% of rows), flagged {WARN_MISSED}.\n"
"The sample index still tracks chip time -- the gaps are " "The gaps are explicit, so the sample index still tracks chip "
"explicit, not compressed --\nbut those rows carry no field " "time, and those\nrows carry no field data. Lower the rate "
"data. Lower the rate (higher --tmrc) or raise --bus-speed\n" "(higher --tmrc) or raise --bus-speed\nto remove them.",
"to remove them.", file=sys.stderr) file=sys.stderr)
return 1 if failed else 0 return 1 if failed else 0
return 0 return 0

50
plot.py
View file

@ -4,10 +4,9 @@
./.venv/bin/python plot.py capture_60s.csv ./.venv/bin/python plot.py capture_60s.csv
./.venv/bin/python plot.py capture_60s.csv -o out.png ./.venv/bin/python plot.py capture_60s.csv -o out.png
Small multiples rather than one shared axis: the three axes sit at very Small multiples, since the three axes sit at very different DC offsets and a
different DC offsets, so a single scale would flatten the variation that shared scale would flatten the variation that matters. Each panel has its own
matters. Each panel therefore has its own y-scale -- read the panels y-scale, with a mean/sd annotation for context.
independently, and note the per-panel mean/sd annotation for context.
""" """
import argparse import argparse
@ -20,24 +19,15 @@ import numpy as np
import calibrate import calibrate
import capture import capture
import characterize
from characterize import GRID, SURFACE, TEXT_PRIMARY, TEXT_SECONDARY
# Light-mode design tokens. # The three peer axes take characterize's categorical slots, so a capture keeps
SURFACE = "#fcfcfb" # the same colours in every figure. The norm is derived from them, so it takes
TEXT_PRIMARY = "#0b0b0b" # neutral ink.
TEXT_SECONDARY = "#52514e" SERIES = [(key, f"{name} axis", color)
GRID = "#e3e2df" for key, name, color in characterize.AXES]
SERIES.append(("total", "Norm |B| = sqrt(X^2 + Y^2 + Z^2)", TEXT_PRIMARY))
# Categorical slots 1-3 for the three peer axes. Validated all-pairs in light
# mode (worst CVD dE 9.2, normal-vision 24.0). The norm is a derived quantity,
# not a fourth peer, so it takes neutral ink instead of a competing hue -- which
# also keeps the categorical set at the three slots that validate for small
# multiples.
SERIES = [
("x", "X axis", "#2a78d6"),
("y", "Y axis", "#eb6834"),
("z", "Z axis", "#1baf7a"),
("total", "Norm |B| = sqrt(X^2 + Y^2 + Z^2)", TEXT_PRIMARY),
]
def rolling_mean(v, window): def rolling_mean(v, window):
@ -45,7 +35,7 @@ def rolling_mean(v, window):
A plain convolution tapers toward zero at the edges. Dividing by the number A plain convolution tapers toward zero at the edges. Dividing by the number
of samples that actually contributed gives a true partial-window mean of samples that actually contributed gives a true partial-window mean
instead, so the ends carry no artefact. instead, so the ends carry no artifact.
""" """
if window < 2: if window < 2:
return v return v
@ -66,15 +56,13 @@ def main():
args = ap.parse_args() args = ap.parse_args()
try: try:
# Either format: a raw capture, or one already converted to nT by # Either format: a raw capture, or one already converted to nT.
# calibrate.py. Which it is, is a property of the file.
cap = calibrate.load_any(args.csv) cap = calibrate.load_any(args.csv)
except (OSError, capture.CaptureError, except (OSError, capture.CaptureError,
calibrate.CalibrationError) as exc: calibrate.CalibrationError) as exc:
sys.exit(str(exc)) sys.exit(str(exc))
# Drift-corrected time: the nominal grid would be ~6% off real seconds. t = cap.elapsed # drift-corrected; the nominal grid is ~6% off
t = cap.elapsed
series = dict(cap.axes()) series = dict(cap.axes())
duration = cap.duration duration = cap.duration
rate = cap.true_rate_hz rate = cap.true_rate_hz
@ -87,8 +75,7 @@ def main():
v = series[key] v = series[key]
ax.set_facecolor(SURFACE) ax.set_facecolor(SURFACE)
# Raw trace kept thin and translucent: at ~376 Hz there are far more # Thin and translucent, there being far more samples than pixels.
# samples than pixels, so a full-weight line would read as a solid band.
ax.plot(t, v, color=color, linewidth=0.4, alpha=0.30, ax.plot(t, v, color=color, linewidth=0.4, alpha=0.30,
solid_capstyle="round", rasterized=True) solid_capstyle="round", rasterized=True)
if window > 1: if window > 1:
@ -96,8 +83,7 @@ def main():
solid_capstyle="round") solid_capstyle="round")
ax.set_ylabel("nT", color=TEXT_SECONDARY, fontsize=10) ax.set_ylabel("nT", color=TEXT_SECONDARY, fontsize=10)
# Direct label instead of a legend: one series per panel, so the title # Direct label: one series per panel.
# names it. This is also the relief the palette's contrast WARN requires.
ax.set_title(label, color=TEXT_PRIMARY, fontsize=12, loc="left", ax.set_title(label, color=TEXT_PRIMARY, fontsize=12, loc="left",
pad=8, fontweight="bold") pad=8, fontweight="bold")
ax.annotate(f"mean {v.mean():,.0f} nT sd {v.std():.0f} nT " ax.annotate(f"mean {v.mean():,.0f} nT sd {v.std():.0f} nT "
@ -123,8 +109,8 @@ def main():
if window > 1 else "") if window > 1 else "")
fig.suptitle("RM3100 magnetometer capture", color=TEXT_PRIMARY, fig.suptitle("RM3100 magnetometer capture", color=TEXT_PRIMARY,
fontsize=15, fontweight="bold", x=0.5, y=0.985) fontsize=15, fontweight="bold", x=0.5, y=0.985)
# A calibrated file has no nominal rate to be wrong against -- it has a # A calibrated file carries a gain factor, which is what is worth stating
# gain factor instead, which is the thing worth stating about it. # about it; a raw one carries a nominal rate to compare against.
provenance = (cap.describe() if hasattr(cap, "describe") else provenance = (cap.describe() if hasattr(cap, "describe") else
f"nominal {cap.nominal_rate_hz:g} Hz, " f"nominal {cap.nominal_rate_hz:g} Hz, "
f"{cap.rate_error * 100:+.1f}%") f"{cap.rate_error * 100:+.1f}%")

100
rm3100.py
View file

@ -46,22 +46,21 @@ STATUS_DRDY = 0x80
EXPECTED_REVID = 0x22 EXPECTED_REVID = 0x22
# 100 cycle count gives a good balance of sensor duty cycle, spectrum, and post-filter noise floor. # Balances duty cycle, spectrum and post-filter noise floor; see the README for
# This should be raised if data rates are higher than desired or bus speed / host speed can't support. # the trade against 50 and 200.
# Lower than 50 is not recommended under any circumstances. See the README for more info.
DEFAULT_CYCLE_COUNT = 100 DEFAULT_CYCLE_COUNT = 100
# Not a hard floor -- the manual's is MIN_CYCLE_COUNT below -- but the point # The point under which the duty cycle starts costing real noise floor, so
# under which the dither margin is too thin to defend, so logger.py warns. # logger.py warns. MIN_CYCLE_COUNT below is the hard floor.
RECOMMENDED_MIN_CYCLE_COUNT = 50 RECOMMENDED_MIN_CYCLE_COUNT = 50
# Table 3-1 tabulates gain and noise up to a cycle count of 400. Past that both # Table 3-1 tabulates gain and noise up to a cycle count of 400. Past that both
# _gain_lsb_per_tesla and expected_noise_nt are extrapolating their fits. # _gain_lsb_per_tesla and expected_noise_nt are extrapolating their fits.
MAX_SPEC_CYCLE_COUNT = 400 MAX_SPEC_CYCLE_COUNT = 400
# The fastest rate register. Used by default so the cycle count, not TMRC, sets # The fastest rate register, used by default so the cycle count sets the rate.
# the rate -- TMRC offers only factor-of-two steps and has no effect at all once # TMRC offers factor-of-two steps, and once the cycle count governs it has no
# the cycle count governs. # effect at all.
TMRC_FASTEST = 0x92 TMRC_FASTEST = 0x92
@ -86,8 +85,7 @@ def gain_lsb_per_tesla(cycle_count):
def tesla_per_count(cycle_count): def tesla_per_count(cycle_count):
"""Calibration constant: multiply a raw count by this to get tesla. """Calibration constant: multiply a raw count by this to get tesla.
Stored in capture headers, so a reader needs no knowledge of the gain fit -- Stored in capture headers, so a reader needs no knowledge of the gain fit.
and it is a multiply at the point of use rather than a divide.
""" """
return 1.0 / gain_lsb_per_tesla(cycle_count) return 1.0 / gain_lsb_per_tesla(cycle_count)
@ -96,41 +94,27 @@ def tesla_per_count(cycle_count):
# #
# per-axis time = cycle_count / COUNTS_PER_SECOND + AXIS_OVERHEAD_S # per-axis time = cycle_count / COUNTS_PER_SECOND + AXIS_OVERHEAD_S
# #
# COUNTS_PER_SECOND comes from the specification, not from a fit: Table 3-1
# gives a 180 kHz circuit oscillation frequency, and section 4.1 measures each
# cycle count in *both* bias directions, so one count costs two oscillations.
# Rates measured here across cycle counts 229..29769 agree with that figure to
# within 0.8%, which confirms the spec value rather than improving on it -- the
# residual is this particular unit's oscillator sitting inside ordinary
# component tolerance, and another part would sit somewhere else.
#
# The overhead has no specified value and must be measured; 68.7 us reproduces
# the observed rates to better than 1% when the divisor is held at spec. It is
# why the naive "rate x cycle_count" constant is not constant, drifting from
# 84,429 at cycle count 100 to 89,191 at 1200.
#
# **This pair is a nominal model, not a description of any particular part.** # **This pair is a nominal model, not a description of any particular part.**
# Solving it freely against one unit's two cycle counts gave 88,546 counts/s # Both terms are per-unit and move with the supply, because the oscillator does,
# with a 40.6 us overhead at 3.006 V, and 92,889 with 38.1 us at 3.353 V -- so # and the resulting error is cycle-count dependent, which makes
# both terms are per-unit *and* move with the supply, because the oscillator # `Capture.rate_error` a configuration diagnostic. calibrate.py holds a measured
# does. Holding the divisor at spec and solving for the overhead alone cannot # per-unit pair; `sweep.py --calibration-out` produces one.
# absorb it: that gives 58.9 us at cycle count 100 and 113.6 us at 400 on the
# same data, fitting neither.
# #
# The consequence to know about is that the error is *cycle-count dependent*. # COUNTS_PER_SECOND is specified rather than fitted: Table 3-1 gives a 180 kHz
# On that unit the model was +0.84% fast at cycle count 100 and -0.99% slow at # circuit oscillation and section 4.1 measures each cycle count in *both* bias
# 400 -- same supply, opposite signs -- so `Capture.rate_error` measured against # directions, so one count costs two oscillations. The overhead has no specified
# it is not a gain indicator. calibrate.py holds the per-unit pair, and # value; 68.7 us is what Table 3-1's own maximum single-axis rates imply. It is
# `sweep.py --calibration-out` measures one. # why the naive "rate x cycle_count" constant is not constant.
COUNTS_PER_SECOND = 90000.0 # 180 kHz (Table 3-1) / 2 bias directions COUNTS_PER_SECOND = 90000.0 # 180 kHz (Table 3-1) / 2 bias directions
AXIS_OVERHEAD_S = 68.7e-6 # nominal; measured parts run 38-41 us AXIS_OVERHEAD_S = 68.7e-6 # nominal; measured parts run 38-41 us
AXES = 3 AXES = 3
# Rates predicted from the model are good to roughly this much on a given unit, # How far a rate predicted from the nominal model can sit from the rate actually
# and no better across units, since the manual quotes no tolerance on the # achieved: the *model's* error envelope. The ~7% of section 5.2.1 is a spread
# oscillator. Anything needing the real number measures it: logger.py calibrates # on the TMRC update rates of Table 5-4, which is a different quantity from the
# the period against the host clock before recording. # cycle-count ceiling that governs here. Anything needing the real number
RATE_TOLERANCE = 0.02 # measures it: logger.py calibrates the period against the host clock.
RATE_TOLERANCE = 0.06
# Table 3-1 quotes 30/20/15 nT at cycle counts 50/100/200, which fits # Table 3-1 quotes 30/20/15 nT at cycle counts 50/100/200, which fits
# K/sqrt(cycle_count). Extrapolation past ~400 is unverified: the manual calls # K/sqrt(cycle_count). Extrapolation past ~400 is unverified: the manual calls
@ -157,20 +141,18 @@ def cycle_count_for_rate(rate_hz, axes=AXES):
"""Cycle count that makes the sensor free-run at about rate_hz. """Cycle count that makes the sensor free-run at about rate_hz.
Inverse of sample_period(). Use with a TMRC faster than the target so the Inverse of sample_period(). Use with a TMRC faster than the target so the
cycle count governs -- then this is a continuous rate knob, where TMRC only cycle count governs. It is then a continuous rate knob, where TMRC offers
offers factor-of-two steps, and the duty cycle is ~100% by construction. factor-of-two steps, and the duty cycle is ~100% by construction.
The achieved rate lands within about RATE_TOLERANCE of the target.
The achieved rate will sit within about RATE_TOLERANCE of the target, set by
oscillator tolerance rather than by this calculation. Measure it if it
matters.
""" """
cc = round((1.0 / (rate_hz * axes) - AXIS_OVERHEAD_S) * COUNTS_PER_SECOND) cc = round((1.0 / (rate_hz * axes) - AXIS_OVERHEAD_S) * COUNTS_PER_SECOND)
return max(1, min(0xFFFF, cc)) return max(1, min(0xFFFF, cc))
# Section 5.1: "quantization issues generally dictate working above a cycle # Section 5.1: "quantization issues generally dictate working above a cycle
# count value of ~30". Below that the LSB grows faster than the sensor's own # count value of ~30". Kept as the manual's floor, though what actually gives
# noise, so the quantiser stops being dithered and averaging stalls. # way below 50 is the duty cycle: the per-axis overhead is a fixed cost against
# a shrinking integration time, and post-filter noise goes as 1/sqrt(duty).
MIN_CYCLE_COUNT = 30 MIN_CYCLE_COUNT = 30
MAX_CYCLE_COUNT = 0xFFFF MAX_CYCLE_COUNT = 0xFFFF
MIN_RATE_BY_CYCLE_COUNT = 1.0 / (AXES * (MAX_CYCLE_COUNT / COUNTS_PER_SECOND MIN_RATE_BY_CYCLE_COUNT = 1.0 / (AXES * (MAX_CYCLE_COUNT / COUNTS_PER_SECOND
@ -186,10 +168,8 @@ def plan(rate_hz=None, cycle_count=None, tmrc=None):
cycle count sets how long a measurement takes, TMRC sets how often one is cycle count sets how long a measurement takes, TMRC sets how often one is
started. Leaving TMRC faster than the cycle-count ceiling means the sensor started. Leaving TMRC faster than the cycle-count ceiling means the sensor
free-runs at ~100% duty; setting it slower makes the sensor idle, which free-runs at ~100% duty; setting it slower makes the sensor idle, which
costs sensitivity for nothing unless low power is the goal. costs sensitivity for nothing unless low power is the goal. So the default
in every branch is TMRC_FASTEST and the cycle count is the rate knob.
So the default in every branch is TMRC_FASTEST, and the cycle count is the
rate knob -- it is continuous where TMRC offers only factor-of-two steps.
""" """
notes = [] notes = []
if cycle_count is None: if cycle_count is None:
@ -269,10 +249,9 @@ class RM3100:
Returns (data, mono, wall): the bus stamps the clocks when the reply Returns (data, mono, wall): the bus stamps the clocks when the reply
lands, and the stamp travels with its own data. lands, and the stamp travels with its own data.
Prefers a single combined transaction (repeated START) where the bus Prefers a combined transaction (repeated START) where the bus offers
offers one, since the host round trip is what limits the sample rate. one, since the host round trip is what limits the sample rate. Falls
Falls back to the manual's STOP-then-START form (sections 4.5.2, 5.8.4) back to the manual's STOP-then-START form (sections 4.5.2, 5.8.4).
for a bus that cannot do it.
""" """
combined = getattr(self.bus, "write_read", None) combined = getattr(self.bus, "write_read", None)
if combined is not None: if combined is not None:
@ -311,9 +290,8 @@ class RM3100:
def configure(self): def configure(self):
"""Put DRDY into a state where polling STATUS actually works. """Put DRDY into a state where polling STATUS actually works.
Verified rather than assumed: exactly-once sampling relies on DRC1=1 The readback is verified here, since exactly-once sampling relies on
clearing DRDY when the measurement registers are read, which is what DRC1=1 clearing DRDY only when the measurement registers are read.
makes a second read of the same measurement impossible.
""" """
self.write_reg(REG_HSHAKE, [HSHAKE_DRDY_ON_READ_ONLY]) self.write_reg(REG_HSHAKE, [HSHAKE_DRDY_ON_READ_ONLY])
# Bits 4-6 are read-only NACK status, so compare only the writable ones. # Bits 4-6 are read-only NACK status, so compare only the writable ones.
@ -355,10 +333,10 @@ class RM3100:
time.sleep(interval) time.sleep(interval)
def read_raw(self): def read_raw(self):
"""Return ((x, y, z) counts, mono, wall) -- the fast path. """Return ((x, y, z) counts, mono, wall): the fast path.
Callers logging at high rates should use this and defer the microtesla Callers logging at high rates use this and defer the unit conversion, so
conversion, so the sampling loop does I2C and nothing else. the sampling loop does I2C and nothing else.
""" """
data, mono, wall = self.read_reg(REG_MX, 9) data, mono, wall = self.read_reg(REG_MX, 9)
return decode_measurements(data), mono, wall return decode_measurements(data), mono, wall

42
sweep.py
View file

@ -5,11 +5,10 @@
./.venv/bin/python sweep.py --from 8 --to 128 ./.venv/bin/python sweep.py --from 8 --to 128
./.venv/bin/python sweep.py --rates 10,25,50 ./.venv/bin/python sweep.py --rates 10,25,50
Cycle count is the rate knob, not TMRC. TMRC only offers factor-of-two steps Cycle count is the rate knob, not TMRC: TMRC offers only factor-of-two steps and
and, once the cycle count governs, has no effect at all -- measured 73.85 Hz at has no effect at all once the cycle count governs. So each point runs with TMRC
TMRC 0x92 against 73.86 Hz at 0x94 for the same cycle count. So each point runs set fast and the cycle count from rm3100.cycle_count_for_rate(), which makes the
with TMRC set fast and the cycle count chosen from rm3100.cycle_count_for_rate(), rate continuous and the duty cycle ~100% by construction.
which makes the rate continuous and the duty cycle ~100% by construction.
Each row is measured, not predicted: the logger calibrates the true period Each row is measured, not predicted: the logger calibrates the true period
against the host clock before recording, and that figure is what is reported. against the host clock before recording, and that figure is what is reported.
@ -33,7 +32,7 @@ MISSED = re.compile(r"WARNING: ([\d,]+) measurement\(s\) were lost")
AMBIGUOUS = re.compile(r"WARNING: ([\d,]+) gap\(s\) could not be counted") AMBIGUOUS = re.compile(r"WARNING: ([\d,]+) gap\(s\) could not be counted")
# Each sweep point is a short capture, so the oscillator barely drifts within # Each sweep point is a short capture, so the oscillator barely drifts within
# one -- but the points are minutes apart and the chip warms through the sweep. # one, and the points are minutes apart while the chip warms through the sweep.
# This is the fractional period uncertainty that implies, and it is what the # This is the fractional period uncertainty that implies, and it is what the
# timing fit's error bars are built on. # timing fit's error bars are built on.
PERIOD_SD = 500e-6 PERIOD_SD = 500e-6
@ -65,10 +64,8 @@ def measure(rate, bus_speed, tmrc, output, record_seconds):
out = proc.stdout + proc.stderr out = proc.stdout + proc.stderr
found = CALIBRATED.search(out) found = CALIBRATED.search(out)
if found: if found:
# Both warnings matter and they are independent. Reporting only losses # Both warnings matter and are independent: an ambiguous gap means the
# would hide the worse case: an ambiguous gap means the sample index # sample index itself may have slipped, even where nothing was lost.
# itself may have slipped, so the timeline is suspect even where no
# measurement was lost.
notes = [] notes = []
for pattern, label in ((MISSED, "lost"), (AMBIGUOUS, "ambiguous")): for pattern, label in ((MISSED, "lost"), (AMBIGUOUS, "ambiguous")):
hit = pattern.search(out) hit = pattern.search(out)
@ -83,11 +80,10 @@ def measure(rate, bus_speed, tmrc, output, record_seconds):
def write_calibration(measured, path, note): def write_calibration(measured, path, note):
"""Fit the timing model across the sweep and save it as a calibration. """Fit the timing model across the sweep and save it as a calibration.
The sweep already measures what a calibration is made of -- a rate at each The sweep already measures what a calibration is made of: a rate at each of
of several cycle counts -- so fitting it here is the difference between a several cycle counts. The reference oscillator is set to the fitted value,
per-unit reference that was measured and one that was assumed. The reference which makes this sweep the definition of "uncorrected" and every later
oscillator is set to the fitted value, which makes this sweep the definition capture correct relative to it.
of "uncorrected" and every later capture correct relative to it.
""" """
if len({cc for cc, _ in measured}) < 2: if len({cc for cc, _ in measured}) < 2:
print(f"\nnot writing {path}: the sweep produced " print(f"\nnot writing {path}: the sweep produced "
@ -126,8 +122,8 @@ def write_calibration(measured, path, note):
print(f" fit residual {fit.residual_ppm:.0f} ppm over {fit.points} points") print(f" fit residual {fit.residual_ppm:.0f} ppm over {fit.points} points")
print(" gain_exponent is 1.0, which the oscillator-clocked measurement " print(" gain_exponent is 1.0, which the oscillator-clocked measurement "
"interval predicts,\n and gain_offset_counts is 0.0. Neither is " "interval predicts,\n and gain_offset_counts is 0.0. Neither is "
"measured by a rate sweep -- both need\n a field that does not move " "measured by a rate sweep. Both need\n a field that holds still "
"between cycle counts. See NOTES.md.") "between cycle counts. See Noise_Floor_Testing/NOTES.md.")
return path return path
@ -183,9 +179,8 @@ def main():
noise = rm3100.expected_noise_nt(cc) noise = rm3100.expected_noise_nt(cc)
# Fraction of the period spent integrating: what actually reduces noise. # Fraction of the period spent integrating: what actually reduces noise.
duty = rm3100.integration_time(cc) * hz if hz else None duty = rm3100.integration_time(cc) * hz if hz else None
# A note always trails the row rather than sitting in a cell: "host # A note trails the row: "host cannot sustain" is wider than any
# cannot sustain" is wider than any sensible column and would shove the # sensible column.
# rest of the line out of alignment.
print(row([ print(row([
f"{rate:g} Hz", f"{rate:g} Hz",
f"{hz:.2f} Hz" if hz else "-", f"{hz:.2f} Hz" if hz else "-",
@ -201,12 +196,13 @@ def main():
"measurement: 208/sqrt(cc) nT, fitted to its 30/20/15 nT at 50/100/200.\n" "measurement: 208/sqrt(cc) nT, fitted to its 30/20/15 nT at 50/100/200.\n"
"Past cycle count ~400 the manual gives no data, so those are\n" "Past cycle count ~400 the manual gives no data, so those are\n"
"extrapolation. Measure the real floor with characterize.py.\n" "extrapolation. Measure the real floor with characterize.py.\n"
"duty is integration time against the period -- what reduces noise.\n" "duty is integration time against the period, which is what reduces\n"
"noise.\n"
"It falls at high rates as the fixed per-axis overhead grows relative\n" "It falls at high rates as the fixed per-axis overhead grows relative\n"
"to the integration, and would fall further if TMRC rather than the\n" "to the integration, and would fall further if TMRC rather than the\n"
"cycle count governed, leaving the sensor idle between measurements.\n" "cycle count governed, leaving the sensor idle between measurements.\n"
"bus use is irreducible I2C traffic against the period -- not occupancy,\n" "bus use is irreducible I2C traffic against the period. Raw occupancy\n"
"which approaches 100% because the loop polls continuously for DRDY.") "approaches 100%, the loop polling continuously for DRDY.")
if args.calibration_out: if args.calibration_out:
write_calibration(measured, args.calibration_out, args.note) write_calibration(measured, args.calibration_out, args.note)

97
tests/test_analysis.py
View file

@ -5,6 +5,7 @@ so a normalisation slip -- the easy mistake in Welch and Allan code, and an
invisible one on real data -- shows up as a factor rather than a wobble. invisible one on real data -- shows up as a factor rather than a wobble.
""" """
import math
from pathlib import Path from pathlib import Path
import numpy as np import numpy as np
@ -388,3 +389,99 @@ def test_make_time_base_declines_a_constant_field(tmp_path):
cap = _drifted(tmp_path / "c.csv", noise_nt=0.0) cap = _drifted(tmp_path / "c.csv", noise_nt=0.0)
with pytest.raises(ValueError, match="constant|no lines"): with pytest.raises(ValueError, match="constant|no lines"):
characterize.make_time_base(cap, str(tmp_path / "c_timebase.png")) characterize.make_time_base(cap, str(tmp_path / "c_timebase.png"))
# --------------------------------------------------------------------------
# Measuring the oscillator's drift
# --------------------------------------------------------------------------
def test_window_rates_recovers_a_planted_ramp(tmp_path):
"""The rate falls at the planted rate, in fractional terms per second."""
ppm_per_second = 8.0
cap = _drifted(tmp_path / "d.csv", ppm_per_second=ppm_per_second)
centres, rates = characterize.window_rates(cap, 10.0)
assert len(rates) == 10
# A period ramping up as (1 + r*t) is a rate falling as (1 - r*t).
slope = np.polyfit(centres, rates / rates.mean(), 1)[0]
assert slope == pytest.approx(-ppm_per_second * 1e-6, rel=0.05)
def test_window_rates_is_flat_without_drift(tmp_path):
cap = _drifted(tmp_path / "f.csv")
_, rates = characterize.window_rates(cap, 10.0)
assert np.ptp(rates) / rates.mean() < 1e-9
def test_window_rates_centres_span_the_capture(tmp_path):
cap = _drifted(tmp_path / "c.csv")
centres, rates = characterize.window_rates(cap, 10.0)
assert len(centres) == len(rates)
assert centres[0] == pytest.approx(5.0)
assert centres[-1] == pytest.approx(95.0)
@pytest.mark.parametrize("window", [0.0, -1.0, 60.0, 1e6])
def test_window_rates_declines_impossible_windows(tmp_path, window):
"""Non-positive, or too long to fit two windows in the capture."""
cap = _drifted(tmp_path / "s.csv")
centres, rates = characterize.window_rates(cap, window)
assert len(centres) == 0 and len(rates) == 0
def test_rate_stability_recovers_a_planted_ramp(tmp_path):
"""Peak to peak across eight windows, against the ramp that produced it."""
cap = _drifted(tmp_path / "r.csv", ppm_per_second=8.0)
# Eight windows of 12.5 s: centres 6.25 s and 93.75 s apart, so the spread
# is the ramp over 87.5 s, not over the full 100 s.
expected = 8.0e-6 * cap.duration * (1 - 1 / 8)
assert characterize.rate_stability(cap) == pytest.approx(expected, rel=0.05)
def test_rate_stability_is_zero_on_a_uniform_grid(tmp_path):
assert characterize.rate_stability(_drifted(tmp_path / "u.csv")) < 1e-9
def test_rate_stability_declines_a_capture_too_short_to_window(tmp_path):
cap = _drifted(tmp_path / "t.csv", rows=200)
assert characterize.rate_stability(cap) == 0.0
def test_rate_allan_of_a_linear_drift_rises_with_tau(tmp_path):
"""sigma_y(tau) = |D| * tau / sqrt(2) for a deterministic frequency ramp.
This is the shape the real captures show above a few seconds, and it is what
separates a warming oscillator from a random walk.
"""
ppm_per_second = 8.0
cap = _drifted(tmp_path / "a.csv", ppm_per_second=ppm_per_second)
taus, devs = characterize.rate_allan(cap, [4.0, 8.0, 16.0])
assert list(taus) == [4.0, 8.0, 16.0]
for tau, dev in zip(taus, devs):
assert dev == pytest.approx(ppm_per_second * 1e-6 * tau / math.sqrt(2),
rel=0.05)
def test_rate_allan_falls_with_tau_when_only_jitter_is_present(tmp_path):
"""Independent read jitter averages down; drift does not. Opposite slopes."""
cap = _drifted(tmp_path / "j.csv", jitter_s=2e-3)
taus, devs = characterize.rate_allan(cap, [4.0, 16.0])
assert devs[0] > devs[1] * 2
def test_rate_allan_is_negligible_on_a_uniform_grid(tmp_path):
cap = _drifted(tmp_path / "z.csv")
_, devs = characterize.rate_allan(cap, [4.0, 8.0])
assert np.all(devs < 1e-9)
def test_rate_allan_drops_taus_that_yield_too_few_windows(tmp_path):
"""Two windows give one difference, which is not an estimate of anything."""
cap = _drifted(tmp_path / "d2.csv")
taus, devs = characterize.rate_allan(cap, [10.0, 40.0, 200.0])
assert list(taus) == [10.0] # 40 s gives two windows, 200 s none
assert len(devs) == 1
def test_rate_allan_accepts_an_empty_request(tmp_path):
taus, devs = characterize.rate_allan(_drifted(tmp_path / "e.csv"), [])
assert len(taus) == 0 and len(devs) == 0

241
tests/test_calibrate.py
View file

@ -20,16 +20,47 @@ from synthetic import drifting_times, field_counts, write_capture
FIELD = (-40000.0, 16000.0, 12000.0) FIELD = (-40000.0, 16000.0, 12000.0)
REFERENCE_HZ = 88546.0
REFERENCE_OVERHEAD_S = 40.61e-6
def planted_period(cycle_count, counts_per_second, overhead): def planted_period(cycle_count, counts_per_second, overhead):
"""Period for a capture whose per-axis overhead is a fixed *duration*.
Within one condition this is indistinguishable from the clocked form below,
which is why `fit_timing` -- which only ever sees one condition at a time --
is tested with it.
"""
return rm3100.AXES * (cycle_count / counts_per_second + overhead) return rm3100.AXES * (cycle_count / counts_per_second + overhead)
def make(path, cycle_count=100, counts_per_second=88546.0, overhead=40.61e-6, def planted_period_clocked(cycle_count, counts_per_second,
overhead=REFERENCE_OVERHEAD_S,
reference_hz=REFERENCE_HZ):
"""Period for a capture whose overhead is a fixed *count* of cycles.
The two forms only diverge once a capture runs at a frequency other than the
one the overhead was measured at, and that is exactly the case a calibration
is applied to. This is the model `calibrate.oscillator_hz` inverts: the
overhead's duration goes as 1/f, so a capture at 92,889 counts/s carries
38.71 us of what was 40.61 us at 88,546.
"""
return rm3100.AXES * (cycle_count + overhead * reference_hz) / counts_per_second
def make(path, cycle_count=100, counts_per_second=REFERENCE_HZ,
overhead=REFERENCE_OVERHEAD_S, clocked=False,
rows=2000, mean=FIELD, noise_nt=5.0, seed=0): rows=2000, mean=FIELD, noise_nt=5.0, seed=0):
"""A synthetic capture with a planted timing model.
`clocked` chooses which hypothesis about the overhead the capture obeys; it
makes no difference unless `counts_per_second` differs from REFERENCE_HZ.
"""
period = (planted_period_clocked if clocked else planted_period)
counts = field_counts(rows, mean, cycle_count, noise_nt, seed) counts = field_counts(rows, mean, cycle_count, noise_nt, seed)
return capture.load(write_capture( return capture.load(write_capture(
path, rows=rows, cycle_count=cycle_count, counts=counts, path, rows=rows, cycle_count=cycle_count, counts=counts,
dt=planted_period(cycle_count, counts_per_second, overhead))) dt=period(cycle_count, counts_per_second, overhead)))
def written(tmp_path, **fields): def written(tmp_path, **fields):
@ -147,13 +178,56 @@ def test_the_nominal_overhead_costs_far_more_and_varies_with_cycle_count(tmp_pat
def test_oscillator_refuses_a_calibration_that_cannot_describe_the_capture(tmp_path): def test_oscillator_refuses_a_calibration_that_cannot_describe_the_capture(tmp_path):
"""An overhead longer than the whole per-axis time is not merely wrong.""" """An overhead that rivals the cycle count is not merely wrong."""
cap = make(tmp_path / "c.csv", cycle_count=100) cap = make(tmp_path / "c.csv", cycle_count=100)
absurd = calibrate.calibration(88546.0, axis_overhead_s=1.0) absurd = calibrate.calibration(88546.0, axis_overhead_s=1.0)
with pytest.raises(calibrate.CalibrationError, match="does not describe"): with pytest.raises(calibrate.CalibrationError, match="does not describe"):
calibrate.oscillator_hz(cap, absurd) calibrate.oscillator_hz(cap, absurd)
@pytest.mark.parametrize("cycle_count", [50, 100, 200, 400])
def test_the_overhead_is_carried_as_counts_not_microseconds(tmp_path,
cycle_count):
"""A capture at another frequency is inverted with its own overhead.
The overhead is a fixed count of oscillator cycles, so a condition running
4.9% faster spends 4.9% less time on it. Subtracting the reference
condition's *duration* instead would leave a residual, and one that grows as
the overhead's share of the per-axis time grows -- which is why the low
cycle counts are parametrised here.
"""
cap = make(tmp_path / f"{cycle_count}.csv", cycle_count=cycle_count,
counts_per_second=92889.0, clocked=True)
cal = calibrate.calibration(REFERENCE_HZ, REFERENCE_OVERHEAD_S)
assert calibrate.oscillator_hz(cap, cal).value == pytest.approx(
92889.0, rel=1e-6)
def test_a_second_condition_is_recovered_as_its_own_timing_fit_would(tmp_path):
"""The single-capture inversion agrees with the two-point fit it replaces.
`fit_timing` needs two cycle counts and recovers both terms for a condition
outright; `oscillator_hz` claims to get the frequency from one capture and a
calibration taken elsewhere. Those two routes have to land in the same
place, or a calibration cannot be carried across supplies at all -- which is
the entire premise of the file.
"""
periods = {cc: planted_period_clocked(cc, 92889.0) for cc in (100, 400)}
fit = calibrate.fit_timing(list(periods), list(periods.values()),
period_sd=1e-6)
assert fit.counts_per_second == pytest.approx(92889.0, rel=1e-9)
# The overhead scaled by exactly the frequency ratio, which is the claim.
assert fit.axis_overhead_s == pytest.approx(
REFERENCE_OVERHEAD_S * REFERENCE_HZ / 92889.0, rel=1e-9)
cal = calibrate.calibration(REFERENCE_HZ, REFERENCE_OVERHEAD_S)
for cycle_count in periods:
cap = make(tmp_path / f"{cycle_count}.csv", cycle_count=cycle_count,
counts_per_second=92889.0, clocked=True)
assert calibrate.oscillator_hz(cap, cal).value == pytest.approx(
fit.counts_per_second, rel=1e-6)
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------
# gain_factor # gain_factor
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------
@ -167,17 +241,20 @@ def test_a_capture_at_its_own_reference_is_not_corrected(tmp_path):
def test_gain_factor_is_the_oscillator_ratio_at_exponent_one(tmp_path): def test_gain_factor_is_the_oscillator_ratio_at_exponent_one(tmp_path):
overhead = 40.61e-6 overhead = REFERENCE_OVERHEAD_S
cap = make(tmp_path / "c.csv", counts_per_second=92889.0, overhead=overhead) cap = make(tmp_path / "c.csv", counts_per_second=92889.0, overhead=overhead,
cal = calibrate.calibration(88546.0, overhead, reference_oscillator_hz=88546.0) clocked=True)
cal = calibrate.calibration(REFERENCE_HZ, overhead,
reference_oscillator_hz=REFERENCE_HZ)
assert calibrate.gain_factor(cap, cal).value == pytest.approx( assert calibrate.gain_factor(cap, cal).value == pytest.approx(
92889.0 / 88546.0, rel=1e-4) 92889.0 / REFERENCE_HZ, rel=1e-4)
def test_a_zero_exponent_disables_the_oscillator_term(tmp_path): def test_a_zero_exponent_disables_the_oscillator_term(tmp_path):
overhead = 40.61e-6 overhead = REFERENCE_OVERHEAD_S
cap = make(tmp_path / "c.csv", counts_per_second=92889.0, overhead=overhead) cap = make(tmp_path / "c.csv", counts_per_second=92889.0, overhead=overhead,
cal = calibrate.calibration(88546.0, overhead, gain_exponent=0.0) clocked=True)
cal = calibrate.calibration(REFERENCE_HZ, overhead, gain_exponent=0.0)
assert calibrate.gain_factor(cap, cal).value == 1.0 assert calibrate.gain_factor(cap, cal).value == 1.0
@ -277,35 +354,6 @@ def test_the_oscillator_term_scales_with_the_exponent(tmp_path):
assert b.relative == pytest.approx(2 * a.relative, rel=1e-9) assert b.relative == pytest.approx(2 * a.relative, rel=1e-9)
def test_rate_stability_is_zero_for_a_constant_period(tmp_path):
"""Not exactly zero: the capture format stores host time to a microsecond,
which is ~0.05 ppm of scatter here. Real drifts run 186 to 1241 ppm, so a
1 ppm bar separates "nothing" from anything worth reporting."""
cap = make(tmp_path / "c.csv", rows=4000)
assert calibrate.rate_stability(cap) < 1e-6
def test_rate_stability_recovers_a_planted_ramp(tmp_path):
"""The oscillator warms up and slows; this is what measures that."""
rows, dt, ramp = 8000, 1 / 300.0, 0.002 # 2000 ppm across the run
index = np.arange(rows)
# Period growing linearly, so the instantaneous rate falls by `ramp`.
times = np.cumsum(dt * (1 + ramp * index / rows))
path = tmp_path / "c.csv"
write_capture(path, rows=rows, dt=dt)
text = path.read_text().splitlines()
header = [l for l in text if l.startswith("#")] + [text[len(
[l for l in text if l.startswith("#")])]]
body = text[len(header):]
rebuilt = header + [
",".join([row.split(",")[0], f"{1_700_000_000.0 + t:.6f}"]
+ row.split(",")[2:])
for row, t in zip(body, times)]
path.write_text("\n".join(rebuilt) + "\n")
cap = capture.load(path)
assert calibrate.rate_stability(cap) == pytest.approx(ramp, rel=0.15)
def test_a_two_point_fit_refuses_to_invent_a_confidence(): def test_a_two_point_fit_refuses_to_invent_a_confidence():
"""It has no residual, so it cannot estimate its own uncertainty.""" """It has no residual, so it cannot estimate its own uncertainty."""
ccs = [100, 400] ccs = [100, 400]
@ -527,8 +575,8 @@ def test_a_calibrated_file_is_not_a_capture(tmp_path):
def test_the_header_records_the_calibration_that_was_applied(tmp_path): def test_the_header_records_the_calibration_that_was_applied(tmp_path):
source = tmp_path / "raw.csv" source = tmp_path / "raw.csv"
cap = make(source, counts_per_second=92889.0, overhead=40.61e-6) cap = make(source, counts_per_second=92889.0, clocked=True)
cal = calibrate.calibration(88546.0, 40.61e-6) cal = calibrate.calibration(REFERENCE_HZ, REFERENCE_OVERHEAD_S)
cal_path = tmp_path / "bench.json" cal_path = tmp_path / "bench.json"
calibrate.save_calibration(cal_path, cal) calibrate.save_calibration(cal_path, cal)
out = tmp_path / "out.csv" out = tmp_path / "out.csv"
@ -595,114 +643,3 @@ def test_cli_reports_a_bad_calibration_rather_than_writing_output(tmp_path):
with pytest.raises(SystemExit, match="rm3100_calibration"): with pytest.raises(SystemExit, match="rm3100_calibration"):
calibrate.main() calibrate.main()
assert not out.exists() assert not out.exists()
# --------------------------------------------------------------------------
# Measuring the oscillator's drift
# --------------------------------------------------------------------------
DRIFT_DT = 1.0 / 250.0
# 102 s: ten 10 s windows with room to spare. Landing exactly on a multiple of
# the window would make the window count depend on whether the planted drift
# happens to push the duration over it.
DRIFT_ROWS = 25_500
def drifting(path, ppm_per_second=0.0, jitter_s=0.0, rows=DRIFT_ROWS,
dt=DRIFT_DT, seed=0):
"""A capture whose sample period ramps, with optional host-side jitter."""
return capture.load(write_capture(
path, rows=rows, dt=dt, counts=field_counts(rows, FIELD, 100, 5.0, seed),
times=drifting_times(rows, dt, ppm_per_second, jitter_s, seed)))
def test_window_rates_recovers_a_planted_ramp(tmp_path):
"""The rate falls at the planted rate, in fractional terms per second."""
ppm_per_second = 8.0
cap = drifting(tmp_path / "d.csv", ppm_per_second=ppm_per_second)
centres, rates = calibrate.window_rates(cap, 10.0)
assert len(rates) == 10
# A period ramping up as (1 + r*t) is a rate falling as (1 - r*t).
slope = np.polyfit(centres, rates / rates.mean(), 1)[0]
assert slope == pytest.approx(-ppm_per_second * 1e-6, rel=0.05)
def test_window_rates_is_flat_without_drift(tmp_path):
cap = drifting(tmp_path / "f.csv")
_, rates = calibrate.window_rates(cap, 10.0)
assert np.ptp(rates) / rates.mean() < 1e-9
def test_window_rates_centres_span_the_capture(tmp_path):
cap = drifting(tmp_path / "c.csv")
centres, rates = calibrate.window_rates(cap, 10.0)
assert len(centres) == len(rates)
assert centres[0] == pytest.approx(5.0)
assert centres[-1] == pytest.approx(95.0)
@pytest.mark.parametrize("window", [0.0, -1.0, 60.0, 1e6])
def test_window_rates_declines_impossible_windows(tmp_path, window):
"""Non-positive, or too long to fit two windows in the capture."""
cap = drifting(tmp_path / "s.csv")
centres, rates = calibrate.window_rates(cap, window)
assert len(centres) == 0 and len(rates) == 0
def test_rate_stability_recovers_a_planted_ramp(tmp_path):
"""Peak to peak across eight windows, against the ramp that produced it."""
cap = drifting(tmp_path / "r.csv", ppm_per_second=8.0)
# Eight windows of 12.5 s: centres 6.25 s and 93.75 s apart, so the spread
# is the ramp over 87.5 s, not over the full 100 s.
expected = 8.0e-6 * cap.duration * (1 - 1 / 8)
assert calibrate.rate_stability(cap) == pytest.approx(expected, rel=0.05)
def test_rate_stability_is_zero_on_a_uniform_grid(tmp_path):
assert calibrate.rate_stability(drifting(tmp_path / "u.csv")) < 1e-9
def test_rate_stability_declines_a_capture_too_short_to_window(tmp_path):
cap = drifting(tmp_path / "t.csv", rows=200)
assert calibrate.rate_stability(cap) == 0.0
def test_rate_allan_of_a_linear_drift_rises_with_tau(tmp_path):
"""sigma_y(tau) = |D| * tau / sqrt(2) for a deterministic frequency ramp.
This is the shape the real captures show above a few seconds, and it is what
separates a warming oscillator from a random walk.
"""
ppm_per_second = 8.0
cap = drifting(tmp_path / "a.csv", ppm_per_second=ppm_per_second)
taus, devs = calibrate.rate_allan(cap, [4.0, 8.0, 16.0])
assert list(taus) == [4.0, 8.0, 16.0]
for tau, dev in zip(taus, devs):
assert dev == pytest.approx(ppm_per_second * 1e-6 * tau / math.sqrt(2),
rel=0.05)
def test_rate_allan_falls_with_tau_when_only_jitter_is_present(tmp_path):
"""Independent read jitter averages down; drift does not. Opposite slopes."""
cap = drifting(tmp_path / "j.csv", jitter_s=2e-3)
taus, devs = calibrate.rate_allan(cap, [4.0, 16.0])
assert devs[0] > devs[1] * 2
def test_rate_allan_is_negligible_on_a_uniform_grid(tmp_path):
cap = drifting(tmp_path / "z.csv")
_, devs = calibrate.rate_allan(cap, [4.0, 8.0])
assert np.all(devs < 1e-9)
def test_rate_allan_drops_taus_that_yield_too_few_windows(tmp_path):
"""Two windows give one difference, which is not an estimate of anything."""
cap = drifting(tmp_path / "d2.csv")
taus, devs = calibrate.rate_allan(cap, [10.0, 40.0, 200.0])
assert list(taus) == [10.0] # 40 s gives two windows, 200 s none
assert len(devs) == 1
def test_rate_allan_accepts_an_empty_request(tmp_path):
taus, devs = calibrate.rate_allan(drifting(tmp_path / "e.csv"), [])
assert len(taus) == 0 and len(devs) == 0

6
tests/test_capture.py
View file

@ -75,11 +75,11 @@ def test_rate_error_is_signed_like_the_rate_not_the_period(tmp_path):
assert slow.rate_error == pytest.approx(250.0 / 282.0 - 1, rel=1e-6) assert slow.rate_error == pytest.approx(250.0 / 282.0 - 1, rel=1e-6)
def test_elapsed_bases_differ_by_exactly_the_rate_error(tmp_path): def test_elapsed_is_the_grid_scaled_by_the_measured_period(tmp_path):
cap = capture.load(write_capture(tmp_path / "c.csv")) cap = capture.load(write_capture(tmp_path / "c.csv"))
assert cap.elapsed[0] == 0.0 assert cap.elapsed[0] == 0.0
ratio = cap.elapsed[-1] / cap.elapsed_nominal[-1] assert cap.elapsed[-1] == pytest.approx(cap.sample_index[-1] * cap.dt_true)
assert ratio == pytest.approx(cap.dt_true / cap.dt_nominal) assert cap.dt_true / cap.dt_nominal == pytest.approx(1 / (1 + cap.rate_error))
def test_duration_spans_the_capture(tmp_path): def test_duration_spans_the_capture(tmp_path):