rm3100/NOTES.md

# Measurement notes

The lab notebook for this rig: what was measured, when, on what, and what it
changed. [README.md](README.md) documents how the code works *now*; this file is
the record of how that was arrived at, and it is append-mostly — an entry is
corrected rather than deleted when a later measurement supersedes it.

Every capture cited here predates the current header format and has been
deleted (`*.csv` is gitignored). The numbers stand as measurements; the files
cannot be re-loaded. Re-record before relying on any of it.

**Rigs.** Two hosts appear below. *Bench* is bare metal. *VM* is a
resource-limited virtual machine with virtualised USB passthrough, shared cores
and background load; every host-side ceiling measured there is pessimistic and
labelled as such.

---

## Superseded conclusions

Read this first — these appear as firm recommendations in earlier entries and
are no longer right.

| Earlier conclusion | Superseded by | Now |
|---|---|---|
| "400 kHz measured best; use 400" | three-run repetition at each speed | 750 kHz is fastest and is the default. The 4% edge for 400 kHz sat inside the run-to-run spread of a single short run |
| "Pair cycle count with a TMRC step: 200/0x94, 400/0x95, 800/0x96" | the cycle-count rate model | Hold TMRC fast and use cycle count as the only rate knob. The pairing advice exists only because the rate was being set by TMRC |
| "Abort the run on a missed measurement" | placeholder rows | The run continues and writes a `MISSED` row, keeping `sample_index` contiguous |
| `~84,333 / cc / 3` for the sensor ceiling | fitting overhead separately | `1 / (3 × (cc/90,000 + 68.7 µs))`. The naive constant is not constant — it drifts from 84,429 at cc=100 to 89,191 at cc=1200 |

---

## First working capture

10 s, cycle count 200, TMRC 0x96, sensor stationary on a desk beside a PC.

```
349 samples over 10.02 s -> 34.8 Hz
  X: mean  +6.473 uT   sd 177 nT
  Y: mean +50.914 uT   sd 106 nT
  Z: mean +18.944 uT   sd 396 nT
  |B|: mean 54.709 uT  sd 245 nT
  sample interval: 28.78 ms, sd 1.52 ms
```

- **Rate.** 34.8 Hz against a nominal ~37 Hz is in spec: §5.2.1 quotes roughly
  7% one-sigma tolerance on the update rate.
- **Magnitude.** 54.7 µT sits inside the 25–65 µT band for Earth's field, which
  is the real confirmation that the gain and the int24 decode are right.
- **Noise** is well above the 15 nT Table 3-1 quotes for cycle count 200, and is
  not a driver problem — most likely proximity to a PC and the USB adapter.
- **Drift.** All three axes fell monotonically over the 10 s (Z most, ~1.3 µT),
  so |B| went from 55.5 to 54.3 µT. Consistent with thermal settling after AVDD
  is first powered; coil DC resistance moves 0.4 %/°C (Table 3-3). Allow a
  warm-up before trusting absolute values.

Nothing here is calibrated: these are raw sensor outputs with no hard- or
soft-iron correction, so individual axis values reflect local distortion as much
as Earth's field.

---

## Supply A/B: 3.3 V adapter vs 3.0 V LDO — indistinguishable

Two 60 s captures on the bench, cycle count 200, TMRC 0x96, 400 kHz.

| | 3V3 adapter | 3.0 V LDO |
|---|---|---|
| \|B\| | 40,345 nT | 42,731 nT (**+5.9%**) |
| sd \|B\| absolute | 35.4 nT | 38.5 nT (+8.8%) |
| sd \|B\| **fractional** | **878 ppm** | **900 ppm** (+2.5%) |

**Compare noise fractionally, not absolutely.** The measured scale moved 5.9%
between the runs, and a scale change carries the noise with it, so most of the
absolute sd difference is an artefact. Normalised, the two supplies differ by
2.5% — indistinguishable on single 60 s captures.

That is the expected outcome. Table 3-5 allows 50 mV<sub>PP</sub> of supply
ripple and the LP5907's ~6.5 µV RMS sits roughly 1000× inside it, as did the
adapter rail most likely. Supply quality was never the binding constraint; the
magnetic environment is.

**The 5.9% scale change cannot be attributed to the supply from this data.** The
sensor also moved: the field direction rotated 6.16°, and the per-axis ratios
(X 1.136, Y 1.051, Z 0.870) spread by 26.6%, where a pure gain change would
scale all three identically. |B| is preserved under rotation but not under
translation through a field gradient, so movement and gain are confounded here.

There is still reason to expect 3.0 V to read *more accurately*, but it comes
from the datasheet rather than these captures: Table 3-1 footnote 1 establishes
every specification — including the gain fit behind `tesla_per_count` — at
3.0 V, noting that performance "will differ" elsewhere. At 3.3 V the gain is an
extrapolation off the calibration point.

This experiment is what `compare.py` was written for; it refuses to attribute a
magnitude difference to gain unless both movement checks pass.

### Designing an A/B supply test

- **Clamp the sensor and never touch it.** Swapping supplies must not disturb
  it. This is the single thing that invalidated the first attempt.
- **Interleave A/B/A/B**, never all of A then all of B. Drift over a session is
  large enough to swamp the effect, and only interleaving lets adjacent pairs be
  differenced.
- **Repeat 3–4 times per configuration.** With one run each, a 2.5% difference
  is indistinguishable from run-to-run variation.
- **Let it thermally settle** before each run. Coil resistance moves 0.4 %/°C
  (Table 3-3), so an unsettled sensor drifts through the measurement.
- Record the configuration with `--note`, so the comparison does not depend on
  filenames surviving:

```bash
./.venv/bin/python logger.py --duration 300 --note "3V3"     --output a1.csv
./.venv/bin/python logger.py --duration 300 --note "3V0 LDO" --output b1.csv
./.venv/bin/python compare.py --group note *.csv
```

---

## 60 Hz mains folds into the band

At ~35 Hz the largest spectral feature by far was a line near 9 Hz reaching
100–125 nT/√Hz against a 4–5 nT/√Hz broadband floor — **25× the noise floor**.

It is 60 Hz aliasing, proven by making it move:

| capture | fs | observed peak | predicted 60 Hz alias |
|---|---|---|---|
| 3V3 | 34.7565 Hz | 9.5063 Hz | 9.5129 Hz |
| 3.0 V LDO | 34.5557 Hz | 9.0725 Hz | 9.1114 Hz |
| long run | 34.5783 Hz | 9.1764 Hz | 9.1566 Hz |

A real 9 Hz magnetic signal would sit still. This one moves with the sample
rate: `fs` changed by 0.20 Hz and the peak moved 0.43 Hz — the ×2 sensitivity of
a second-order fold, exactly as predicted.

The RM3100 has **no anti-alias filter** — cycle-count integration is a boxcar
with a poor stopband — so out-of-band interference folds in freely.

**Sampling faster resolves it instead.** At 150 Hz (Nyquist 69.3 Hz) mains lands
in band and the strongest line sits at **59.923 Hz**, observed directly. At
75 Hz it folds to 9.304 Hz against a 9.233 Hz prediction, with only 4 nT/√Hz
left at 60 Hz because that frequency is no longer sampled. This is why the
default configuration runs at ~282 Hz: at that rate mains *and* its second
harmonic are inside the band and can be notched afterwards, which no amount of
filtering can do to an alias.

---

## Overnight run: 13.2 h, 1.63 M samples

Cycle count 200, TMRC 0x96. **Acquisition integrity was perfect** — sample index
contiguous across all 1,635,055 rows, system clock monotonic with no NTP steps,
no malformed rows, no int24 saturation, no missed measurements (worst DRDY
bracket 11.0 ms of 27.03 ms, 41% of margin). Both problems in the data were
outside the logger.

**1. The first ~1.5 h and the last ~1 h are contaminated by handling.** |B|
reaches 198,268 nT — five times Earth's field — in bursts around t+0.44 h.
Hourly |B| sd tells the story:

| hours | \|B\| sd | usable |
|---|---|---|
| 0–1 | 3,750 nT | no — handling |
| 2–10 | **34–40 nT** | yes |
| 11–12 | 41–68 nT | marginal |
| 13 | 247 nT | no — activity resumed |

A 5-minute slice from the middle of the quiet window gave |B| = 42,132 nT with
sd 32.5 nT (771 ppm).

**2. A single rate does not describe a 13-hour capture.** One slope across the
whole run leaves a residual sd of 5.3 s and a 22.8 s span. Hour-by-hour fits
show the oscillator speeding up monotonically:

| hour | fitted period | vs run mean |
|---|---|---|
| 2 | 29.0187 ms | +1254 ppm |
| 6 | 28.9794 ms | −102 ppm |
| 12 | 28.9187 ms | −2195 ppm |

That is ~2,500 ppm of rate variation across the quiet hours — thermal, as
expected of an RC oscillator. So **long captures cannot be given a single
frequency axis**; spectra must be computed on windows short enough that the rate
is constant, via `Capture.restrict()`.

This is what `drift_limited` and the separate reporting of read jitter and fit
residual exist for. Over 5 minutes the residual is 3.7 ms; over 8.5 hours it is
2,356 ms — while read jitter stays 0.21 ms in both. Conflating the two would
have read as a 20,000× worse host.

**Drift-limited beyond a second.** A 769 s window of the same run: per-axis sd
rises to 214–284 nT while the broadband floor barely moves (5.1–5.7 nT/√Hz). The
extra spread is low-frequency drift, not broadband noise — the Allan deviation
bottoms at ~13 nT near τ = 0.12 s and climbs steadily after.

---

## Host-side rate ceilings

Two fixes roughly doubled the achievable rate.

### 1. Two USB round trips per register read → one

Reading a register as a pointer write plus a separate read — the form the manual
draws (§4.5.2, §5.8.4) — costs two round trips. The part accepts a **repeated
START**, so `ch347.write_read()` does it in one. Measured at 400 kHz:

| | separate | combined |
|---|---|---|
| `data_ready()` | 0.652 ms | **0.350 ms** |
| `read_raw()` | 0.838 ms | **0.548 ms** |

### 2. The GIL, not the OS scheduler

Even after that, `logger.py` stalled ~10 ms at 150 Hz while the *same sampling
loop with no writer thread* peaked at 3.1 ms (p99.9 = 2.9 ms, zero threshold
crossings in 2079 samples). The sampler releases the GIL inside each USB
transfer and must re-acquire it, waiting up to Python's default **5 ms** switch
interval while the writer holds it. Setting `sys.setswitchinterval(0.0005)` cut
the worst bracket from 9.96 ms to 5.65 ms and made 150 Hz pass.

Worth remembering generally: with a latency-sensitive loop and a helper thread,
the default GIL switch interval *is* the jitter floor.

### 3. Splitting the writer off the sampling thread

The same 60 s capture yielded 22,558 samples (376 Hz) with inline writing and
28,864 (481 Hz) with the writer split off — about **28%**.

### Measured ceilings (VM, cycle count 50, 400 kHz)

| TMRC | Nominal | Result | Worst DRDY bracket |
|---|---|---|---|
| 0x92 | 600 Hz | fail | 1.86 ms of 1.67 ms (112%) |
| 0x93 | 300 Hz | fail | 3.45 ms of 3.33 ms (104%) |
| 0x94 | 150 Hz | borderline | 5.65 / 5.95 / 6.53 / 7.19 ms of 6.67 ms |
| 0x95 | 75 Hz | clean, 69.2 Hz | 2.63–5.16 ms of 13.33 ms (20–39%) |
| 0x96 | 37 Hz | clean, 34.7 Hz | 3.30–8.56 ms of 27.03 ms (12–32%) |

> **These ceilings are host-specific and pessimistic** — VM, virtualised USB,
> shared cores, load average 2.76 at the time of the one 150 Hz failure. The
> *shape* carries over (the limit is host round-trip latency, not I²C bandwidth
> or the sensor), but the numbers must be re-measured on the machine that will
> run the capture. The "% of margin used" line `logger.py` prints after every
> run is the number to watch.

### The bus is never the constraint

Per sample the bus carries a DRDY poll plus a results read — 150 bit-times —
against a host cost 2–3× larger.

| | 400 kHz | 100 kHz |
|---|---|---|
| bus time per sample | 0.375 ms | 1.50 ms |
| as % of a 28.86 ms period (37 Hz) | **1.3%** | 5.2% |
| as % of a 7.21 ms period (150 Hz) | **5.2%** | 20.8% |
| measured host cost | 0.89–0.98 ms | — |

That puts the bus-limited ceiling near 1100 Hz, roughly double the sensor's own
562 Hz three-axis maximum at cycle count 50 — so absent host stalls, zero misses
is always achievable. Every miss observed has been a host stall.

### Bus speed, three runs each at cycle count 228

| bus | bus time | host cost | implied USB overhead |
|---|---|---|---|
| 100 kHz | 1.500 ms | 2.03–2.19 ms | ~0.63 ms |
| 400 kHz | 0.375 ms | 0.88–0.95 ms | ~0.55 ms |
| **750 kHz** | 0.200 ms | **0.78–0.80 ms** | ~0.59 ms |

Host cost is `bus time + ~0.6 ms` of fixed USB round-trip latency, so a faster
bus helps but cannot go below that floor. An earlier note claimed 400 kHz
benchmarked faster than 750; that rested on a single short run whose 4%
difference was inside the run-to-run spread and does not survive repetition.
400 kHz remains a sane fallback if 750 proves marginal on long wiring.

---

## Timing: how the miss detector was arrived at

**The read-to-read interval is not a usable threshold.** On the VM it reaches
37 ms against a 28.8 ms period — a 29% overshoot from host stalls alone — while
the DRDY bracket stays under 10 ms. The first implementation thresholded the
interval and aborted a perfectly healthy capture at sample 88.

**Period calibration is worth 100×.** Counting how many grid points passed
unseen inside a bracket needs the real period, and the TMRC table value is 6–9%
out on this unit:

| | period | error |
|---|---|---|
| nominal (TMRC table) | 27.03 ms | −6.4% |
| **calibrated (1 s)** | **28.8849 ms** | **0.068%** |
| whole-run fit (25 s) | 28.8653 ms | — |

**Counting needs the interval, detection needs the bracket.** Fit residual after
adopting the two-signal scheme:

| capture | lost | residual |
|---|---|---|
| 37 Hz clean | 0 | **0.019 periods** |
| 150 Hz clean | 0 | 0.082 periods |
| 150 Hz under CPU load | 0 | 0.444 periods |
| 300 Hz | 10 | 1.241 periods |

The load case is the one that shows the design paying off: the bracket reached
119% of threshold, so a stall genuinely occurred, but the interval showed only
one period had elapsed — nothing was lost. The older `int(bracket/period)`
estimator would have inserted a spurious placeholder and slipped the index from
there on. At 300 Hz the ambiguous counter fired 19 times.

**Run-to-run stability.** Fitted periods land 6.0–6.6% below nominal, and two
consecutive 60 s runs agreed to **0.017%**, so a single-slope drift correction
is sound over at least a minute.

---

## Rate sweep

`sweep.py` end to end. Every target from 1 to 128 Hz landed within 0.1%
(columns as of the run; the current tool also prints duty):

```
  target   measured     err     cc    LSB/uT  nT/count    noise  bus use
      1H      1.00H  +0.0%  29769   10929.7     0.091    1.21n    0.0%
      8H      8.01H  +0.1%   3718    1366.4     0.732    3.41n    0.3%
     32H     32.00H  -0.0%    927     341.8     2.926    6.83n    1.2%
    128H    127.99H  -0.0%    229      85.6    11.687   13.75n    4.8%
    256H    249.23H  -2.6%    113      43.0    23.265   19.57n    9.3%  2 lost
```

256 Hz is where the VM gives out; everything below it is exact. Noise past cycle
count ~400 is extrapolation — Table 3-1 ends there.

**Cycle count sets the ceiling, not TMRC.** Measured at 400 kHz:

| Cycle count | Measured | Table 3-1 predicts |
|---|---|---|
| 200 | 152 Hz | 147 Hz |
| 100 | 297 Hz | 283 Hz |
| 50 | 538 Hz | 533 Hz |

**The manual contradicts itself here and Table 3-1 is the correct one.** The
§5.2.1 note claims cycle count 200 allows a "~430 Hz" three-axis update rate;
Table 3-1 gives 440 Hz *single-axis* and says to divide by 3, i.e. ~147 Hz.
Measurement backs Table 3-1 at all three cycle counts.

**When the cycle count governs, TMRC is irrelevant** — cc=400 gave 73.85 Hz at
TMRC 0x92 and 73.86 Hz at 0x94. Varying only cc at TMRC 0x94:

| cc | measured | governed by | duty |
|---|---|---|---|
| 50 | 138.53 Hz | TMRC | 25% |
| 200 | 138.84 Hz | boundary | 99% |
| 400 | 73.83 Hz | cycle count | 100% |
| 800 | 37.09 Hz | cycle count | 100% |

**Idling costs sensitivity.** Reaching ~130 Hz two ways:

| | rate | cc | duty | ASD |
|---|---|---|---|---|
| TMRC-governed (cc=50, 0x94) | 138.5 Hz | 50 | **23%** | 3.42 nT/√Hz |
| cycle-count-governed (cc=228) | 128.5 Hz | 228 | **98%** | 2.39 nT/√Hz |

Same chip, same field, same per-sample physics — the difference is idle time,
and it is why the current model holds TMRC fast and uses cycle count alone.

**Equal duty means equal noise.** Two configurations at ~100% duty and very
different rates:

```
cc200 @150Hz  138.77 Hz  LSB 13.35 nT  ASD 2.98 nT/rtHz  (66% bracket margin)
cc800 @37Hz    34.72 Hz  LSB  3.39 nT  ASD 3.26 nT/rtHz  (10% bracket margin)
```

Near-identical spectral density, as the equal-duty argument predicts — so the
choice between them is bandwidth versus resolution, not noise.

**Low cycle counts are dominated, not fast.** cc=50 reached the same 138.8 Hz as
cc=200 on the VM — both host-limited, not sensor-limited — while idling 75% of
the time and quantising 3.8× more coarsely. Its extra sensor speed is capability
the host cannot collect.

---

## 60 s capture at maximum rate (office rig, historical)

28,864 samples, 481 Hz, cycle count 50, 400 kHz. The sensor was nudged by hand
at t ≈ 41 s, so statistics are split around that:

| Axis | sd, 0–40 s | sd, 47–60 s | step across the event |
|---|---|---|---|
| X | 84 nT | 85 nT | +280 nT |
| Y | 388 nT | 376 nT | +217 nT |
| Z | 187 nT | 198 nT | +232 nT |
| \|B\| | 369 nT | 358 nT | +313 nT |

- **The step was the sensor being physically moved**, confirmed at the time, not
  an electrical artefact. A norm shift does not by itself imply an external
  source changed: |B| is preserved under rotation in a uniform field, but a
  translation through a gradient samples a different local field. Near a PC the
  gradients are steep (§4.2.2: field falls off as 1/distance³), so a few
  centimetres is ample for the +313 nT seen here. Treat a |B| change as "not a
  pure rotation", nothing more.
- **Noise is strongly axis-dependent** — Y is 4.6× X (388 vs 84 nT) and stable
  across the event, so it is not a consequence of the disturbance. The coils are
  nominally identical, so this points at orientation relative to a local noise
  source rather than a sensor fault. X at 84 nT is within ~3× of the 30 nT
  Table 3-1 quotes for cycle count 50; Y is ~13×.
- **Timing was jittery**: intervals averaged 2.08 ms (481 Hz) with sd 1.09 ms
  and a 19.6 ms worst case. This capture predates the chip-grid time base, so it
  is unsuitable for spectral work without resampling. A capture taken today
  carries an exact grid regardless of read jitter.

---

## Supply ripple: what the manual does and does not say

Table 3-5 gives a flat 50 mV<sub>PP</sub> ripple limit on AVDD and DVDD with
**no frequency qualifier**, and no transfer function from ripple to field error
— no µT-per-mV, no PSRR, no ripple-vs-noise curve.

Slow drift should matter less than fast ripple, though this is inference from
the described architecture rather than an explicit claim. §4.1 describes the
output as "the difference in the time to complete the measurement for each
bias" — a differential measurement across forward and reverse coil bias — and §2
claims measurements are "stable over temperature and inherently free from offset
drift". Anything common to both half-measurements largely cancels, so supply
variation slower than one measurement cycle should be rejected far better than
noise near the ~180 kHz LR oscillation.

Unverified on this rig: whether the breakout carries local decoupling, and what
the CH347's 3V3 rail ripple actually measures against the 50 mV limit.

---

## Open questions

- Re-run the LDO comparison with the sensor **clamped**, to separate gain from
  movement. This is the one experiment the A/B entry above failed to perform.
- Deal with 60 Hz coupling at source — standoff distance, then a repeat
  characterisation — rather than only sampling fast enough to see it.
- Check the dither assumption at cycle count 100 against a genuinely quiet site.
  `characterize.py` prints `sd/LSB` per axis; the concern is that a sensor
  quieter than spec has less dither than the 0.79 LSB the spec figure implies.
- Reliability over extended runs on bare metal, where the VM ceilings above do
  not apply.