rm3100/README.md

# RM3100 logging over a CH347 USB-I2C adapter

Logs a PNI RM3100 geomagnetic sensor connected to the I²C pins of a Waveshare
USB to UART/I2C/SPI/JTAG adapter (CH347, USB `1a86:55db`), with a time base good
enough for spectral work: the sample index is a chip-clock grid coordinate, not
a count of host reads.

Reference throughout: *RM3100 & RM2100 Sensor Suite User Manual*, Doc 1017252
R07 (`PNI Sensor - RM3100-Sensor-Suite-User-Manual-R07-1.pdf`, in this
directory). Section numbers below refer to it.

[NOTES.md](NOTES.md) holds the measurement notebook — what was measured on this
rig, and why the design ended up as it is.

## Quickstart

```bash
./setup.sh                                        # udev rule + venv, sudo for the rule only
./.venv/bin/python diagnose-comms.py              # walk the signal chain, stop at the first fault
./.venv/bin/python logger.py --duration 60        # capture to rm3100_<timestamp>.csv
./.venv/bin/python characterize.py rm3100_*.csv   # noise floor, spectrum, stability
```

The test suite needs no hardware:

```bash
./.venv/bin/python -m pytest
```

## Hardware

Waveshare adapter in **Mode 1** (UART1 + I2C + SPI), voltage selector at **3V3**.
The RM3100 is on a breakout board. All connections below are confirmed working.

| Adapter | RM3100 | Notes |
|---|---|---|
| I2C VCC (3V3) | DVDD | digital supply |
| I2C VCC (3V3) | VDD (= AVDD) | analog supply — **required for measurements** |
| I2C VCC (3V3) | I2CEN | **required to select I2C over SPI** |
| I2C GND | DVSS | digital ground |
| I2C SCL | SCK/SCL | pin 27, shared with SPI SCLK |
| I2C SDA | SI/SDA | pin 1, shared with SPI MOSI |

All three 3V3 connections share the adapter's single I2C VCC pin. Current draw is
negligible — ~260 µA per axis at 24 Hz (Table 3-1).

Four pins are worth knowing about; two of them cause failures that are easy to
misdiagnose.

**I2CEN (pin 22) must be tied HIGH.** `HIGH = I2C, LOW = SPI` (§4.3.1). SDA/SCL
are shared with SPI MOSI/SCLK and I2CEN selects between them. Left floating or
low, the chip stays in SPI mode and **never ACKs its I2C address** — the bus scan
finds nothing at all, even though the adapter is perfectly healthy. At
DVDD = 3.3 V it needs `VIH = 0.7 × DVDD` = 2.31 V, so tie it directly to 3V3.

**AVDD/AVSS (pins 4, 5) must be powered to take measurements.** §4.3.1: *"AVDD
can be turned off when not making a measurement to conserve power, since all
other operations are supported with DVDD."* With only DVDD connected the part is
**half alive** in a way that reads as working: the address responds, REVID reads
`0x22`, registers write and read back — but the analog section driving the coils
is dead, so DRDY never asserts and the measurement registers stay at zero. BIST
reporting `XOK=YOK=ZOK=0` is the unambiguous tell. Both rails are needed, and
AVDD is not a substitute for DVDD: DVDD must come up **before or with** AVDD,
never after, and the two must stay within **0.1 V** while running (§4.3.1,
Table 3-5) — which sharing one 3V3 rail satisfies for free.

**SA0 (pin 3) / SA1 (pin 28) set the address.** The top five bits are fixed at
`0b01000`, giving 0x20–0x23 (§4.5). On this breakout both straps are high, so the
sensor answers at **0x23**. The pins are shared with SPI SSN and MISO, so a
breakout may label them `SSN/SA0` and `MISO/SA1`. `logger.py` scans all four
addresses, so no configuration is needed.

**DRDY (pin 23) is not connected.** Not required — the driver polls the STATUS
register instead, which the manual explicitly offers as an alternative (§5.4.1).

### Power

| Parameter | Limit | Source |
|---|---|---|
| Ripple on AVDD or DVDD | 50 mV peak-to-peak | Table 3-5, `VDD_ripple` |
| DVDD − AVDD while running | ±0.1 V | Table 3-5, `ΔVDD_OP` |
| Supply range | 2.0–3.6 V (typ 3.0) | Table 3-5 |
| Absolute maximum | 3.7 V | Table 3-4 |

Reference decoupling (Figures 4-1, 4-2) is a **10 µF bulk capacitor in parallel
with 0.1 µF ceramic**, rail to ground, feeding AVDD, both DVDD pins and I2CEN.
That is the entire filtering specification — no ferrite, no split rails, no LDO
requirement. §4.2.3 adds a placement rule: *"Keep capacitors, especially tantalum
capacitors, far away from the sensor coils"* — a magnetic concern, not an
electrical one.

**Every number in Table 3-1 is quoted at 3.0 V** (footnote 1: *"performance will
differ"*). 3.353 V is comfortably in range, but the gain fit behind
`tesla_per_count` is strictly a 3.0 V figure, so running there is an
extrapolation off the calibration point. **This has now been measured**, on
interleaved captures at 3.006 V and 3.353 V:

| | effect of +0.347 V |
|---|---|
| **Sample rate** | **+4.94%** (repeats to 0.04% across two cycle counts) |
| **Total field \|B\|** | **−7.1%**, about −3,245 nT |
| Broadband noise | none resolvable, ~4% fractionally |

So **supply quality does not set the noise floor, but supply *voltage* sets the
scale**, at roughly −20 %/V. A ±1% regulator at 3.0 V holds scale to ~0.2%; a
USB-derived rail free to move ±5% holds it to ~1%, on top of the ~7% fixed offset
from sitting off the calibration point.

Both effects are the same thing seen twice. The chip's measurement interval is
clocked by its own oscillator — the per-axis overhead is a fixed ~3.6 *counts*,
not a fixed time — so the same number of counts is collected over a window whose
duration goes as 1/f. Gain follows integration time, giving `gain ∝ (cc + n)/f`
with **no free exponent**. That makes the calibrated period every capture already
records a proxy for the rail, correctable without a voltmeter. Full analysis,
including what is *not* settled, in
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).
[NOTES.md](NOTES.md) covers what the manual does and does not say about ripple.

## Setup

```bash
./setup.sh
```

Idempotent. It checks the adapter is present, installs a udev rule, creates
`.venv` with pyusb, numpy, matplotlib and pytest, then verifies the device node
is actually writable and prints whichever fix applies if it is not. `sudo` is
needed for the udev rule and nothing else.

The rule is needed because the CH347's USB node defaults to `root:root 0664`. It
grants access two ways, because no single mechanism covers every distro:

```
# /etc/udev/rules.d/60-ch347.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", TAG+="uaccess"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55db", GROUP="plugdev", MODE="0660"
```

`uaccess` has systemd-logind put an ACL on the node for whoever holds the local
seat — no group, no logout — and is the only mechanism that can work on atomic
Fedora. `GROUP`/`MODE` is the fallback for ssh sessions and seatless systems,
using whichever of `plugdev` or `dialout` exists as a system group.

**Two details in that file are load-bearing**, and getting either wrong produces
a rule that installs cleanly and grants nothing:

- **The prefix must sort below 73.** udev merges every rules directory into one
  lexicographic sequence, and the only thing that acts on the tag is
  `TAG=="uaccess", ENV{MAJOR}!="", RUN{builtin}+="uaccess"` in systemd's
  `73-seat-late.rules`. At `99-` the tag is added after that line has already
  run: set, never read, no ACL. systemd keeps its own uaccess rules in
  `70-uaccess.rules` for this reason.
- **`TAG+="uaccess"` and `GROUP=` must be on separate lines.** udev discards a
  whole rule line whose `GROUP=` it cannot resolve, and since systemd 258 that
  includes any group that exists but is not a *system* group. Sharing a line
  means an unusable group silently takes the uaccess tag down with it.

Together those explain a failure that looks distro-specific but is not: on
Debian and Ubuntu the `plugdev` group grants access on its own, masking a
uaccess tag that never fired. Fedora removed `plugdev` years ago, so on
**Bazzite** neither half worked and the adapter stayed inaccessible.

Two more atomic-Fedora traps `setup.sh` prints fixes for:

- `usermod -aG dialout $USER` fails with *"group 'dialout' does not exist"* even
  though `getent` finds it, because `nss_altfiles` lets `getent` read
  `/usr/lib/group` while `usermod` writes `/etc/group` alone. Copy the line
  across first, then re-run `usermod` and log back in.
- Bazzite is known not to reload `/etc/udev/rules.d` when it switches to the
  final rootfs ([ublue-os/bazzite#2516](https://github.com/ublue-os/bazzite/issues/2516)),
  so a correct rule can sit inert until
  `sudo udevadm control --reload-rules && sudo udevadm trigger` is run once.

`setup.sh` checks the node itself at the end rather than guessing, and says
*which* mechanism granted access — `uaccess ACL` or `group membership` — since
the two fail in different ways and only one of them is available on Bazzite.

Python here is PEP 668 externally-managed, hence the venv rather than a
system-wide `pip install`.

## Usage

```bash
./.venv/bin/python logger.py --duration 10      # log for 10 s
./.venv/bin/python logger.py                    # log until Ctrl-C
./.venv/bin/python logger.py --scan-only        # bus scan only, for wiring checks
```

| Flag | Default | Meaning |
|---|---|---|
| `--rate` | — | target Hz; derives the cycle count and TMRC. Mutually exclusive with `--cycle-count` |
| `--cycle-count` | 100 | sets both the rate and the LSB — see [Configuration](#configuration) |
| `--tmrc` | `0x92` | rate register. The default is the fastest, letting the cycle count set the rate; give one only to sample *slower* |
| `--duration` | 0 | seconds, 0 = until Ctrl-C |
| `--output` | timestamped | CSV path |
| `--address` | autodetect | skip the bus scan |
| `--bus-speed` | 750 | I²C kHz. 100 kHz would spend 42% of each period on the bus at the default cycle count; 750 spends 6% |
| `--calibrate` | 1.0 | seconds of loss-free samples used to measure the true period before recording. The run aborts if no clean stretch can be found |
| `--note` | — | free text recorded in the capture header, e.g. the supply under test |
| `--high-priority` | off | raise the sampling thread to nice −10. Needs `CAP_SYS_NICE`, so run under `sudo`; output files are handed back to the invoking user |
| `--scan-only` | — | scan and exit |

**Exit status is non-zero if the capture is compromised in any way** — a lost
measurement, an ambiguous gap, a truncated run, or an undrained writer. Each
prints an explanation on stderr, and the file is always valid as far as it goes.

Before recording, `logger.py` prints how the configuration was derived, so it can
be checked rather than trusted, then measures the true sample period against the
host clock. Both appear in the capture header.

### Analysis tools

```bash
./.venv/bin/python plot.py capture.csv              # -> capture.png: X, Y, Z, |B|
./.venv/bin/python plot.py capture.csv --smooth 0   # raw trace only
./.venv/bin/python characterize.py capture.csv      # -> capture_noise.png + summary
./.venv/bin/python characterize.py capture.csv --trim 30    # drop settling/handling
./.venv/bin/python compare.py --group note *.csv    # A/B two conditions
./.venv/bin/python compare.py --trim 30 --supply LDO=3.006 --supply 3V3=3.353 \
    LDO/cc100=a.csv 3V3/cc100=b.csv LDO/cc400=c.csv 3V3/cc400=d.csv -o cmp.png
./.venv/bin/python sweep.py --rates 10,50,150       # measure what each rate delivers
```

- **`plot.py`** — four stacked panels with independent y-scales, since the three
  axes sit at very different DC offsets.
- **`characterize.py`** — amplitude spectral density against the 1.2 nT/√Hz of
  Table 3-1, Allan deviation, residual distribution, and host read latency, all
  on **fixed axes** so two runs can be laid side by side. Reports `white sd`
  (`sd(diff)/√2`, which rejects drift) beside the plain `sd`, and a **dither
  check** — whether averaging still gets below one quantiser step. Flags any line
  sitting at a simple fraction of the sample rate. Also writes a
  `_spectrogram.png`: |B| in time and frequency at 0.025 Hz resolution, with
  fs/4 and mains marked — where mains has folded, the line is drawn at the alias
  and labelled as one. Drawn in **ppm of |B| per √Hz** on a fixed colour scale,
  with native bins averaged to a fixed 0.1 Hz step so every capture gets the
  same degrees of freedom per cell; both are needed before one scale across
  several captures means anything. `--spectrogram`, `--overlap`, `--max-freq`
  and `--colormap` tune it; the default `turbo` makes narrow lines legible,
  `viridis` is the lightness-monotonic option.
  Two further figures come out beside it:
  `_drift.png` — the chip's oscillator measured against the host clock, as rate
  against elapsed time with error bars, and beside it the two-sample deviation
  σ_y(τ) that supplies those bars. The minimum of that curve is the best the
  rate can be known and the averaging time worth using; the branch rising ∝ τ to
  its right is real drift, not measurement noise. These parts run 186–1241 ppm
  of warm-up against a floor near 10 ppm at τ ≈ 2–10 s.
  `_timebase.png` — for each detected line, its coherent amplitude computed on
  the chip's sample grid and again on the host clock. A line at a fixed
  frequency sharpens on the host clock; one locked to the sampling is wrecked by
  it. That settles **from a single capture** what `compare.py` otherwise needs
  two different rates to decide. Diagnostic only: no other figure uses the host
  time base, because neither choice is right for both kinds of line.
- **`compare.py`** — everything cross-capture. Reports noise **fractionally**
  (ppm of |B|), because a gain change carries the noise with it and an absolute
  comparison reads a pure scale change as a noise difference. Also: one common
  band for every capture; the timing model solved per condition; whether a line
  is locked to the sampling or to a frequency; whether filtering and decimating a
  fast capture matches a natively slow one; and |B| under each candidate
  correction. Label captures `CONDITION/variant=path` — the part before the `/`
  is the thing under test, and `--supply CONDITION=VOLTS` attaches a rail to it.
- **`sweep.py`** — runs `logger.py` once per target rate and tabulates what each
  configuration actually achieved. `--calibration-out` fits the timing model
  across the sweep and writes it as a calibration.
- **`calibrate.py`** — converts a raw capture to a nanotesla CSV, applying a
  calibration when one is named and nothing when one is not. See below.

`--trim SECONDS` drops that much from *both* ends of a capture. A run usually
opens while the sensor is still settling and closes with a hand on the rig, and
neither end is a noise measurement. Nothing is dropped unless asked, and a
capture shorter than 4× the trim is analysed whole with a note rather than gutted.

### Calibration

**Nothing is corrected by default.** `calibrate.py` without `--calibration`
changes units and nothing else, and the analysis tools read raw captures as they
always did. That is deliberate: the corrections below are real but not yet
settled enough to apply silently.

```bash
./.venv/bin/python sweep.py --rates 32,64,128,256 \
    --calibration-out bench.json --note "unit A at 3.006 V"
./.venv/bin/python calibrate.py raw.csv -o field.csv --calibration bench.json
./.venv/bin/python compare.py --calibration bench.json a.csv b.csv
```

A calibration is **per unit and per supply**, because the chip times everything
from one oscillator and that oscillator is neither the specified frequency nor
fixed. Solving `period = 3 × (cc/C + overhead)` freely against one unit gave
C = 88,546 with a 40.6 µs overhead at 3.006 V, against the nominal 90,000 and
68.7 µs — and C moved to 92,889 at 3.353 V.

That is worth having even if you never correct a field, because it makes the
timing model right: with the measured pair, the predicted rate matches the
measured one to −0.00% at both cycle counts, where the nominal model is 1.9% out
between them. Once it is right, **one capture recovers the oscillator** —
`C = cc / (period/3 − overhead)` — to about 0.09% at cycle count 100.

Two corrections then become available, and neither is on by default:

| | what it is | status |
|---|---|---|
| oscillator gain | `(C / reference) ^ gain_exponent` | −7.1% for +0.347 V. The exponent is **1**, which an oscillator-clocked measurement interval predicts and the measured rates support at 0.3σ against 1.2σ for the alternative |
| cycle-count gain | `gain_offset_counts` | `gain ∝ (cc + n)`. Table 3-1 implies n = 4.086; measured on one unit it is 0.90 and −0.09. Together the two corrections take a 10.3% disagreement to 2.5%, against a 2.3–3.1% movement floor |

Every correction arrives with a propagated standard deviation — 0.3–0.4% on
these captures, well under the residual, which is what says the residual is real
rather than calibration slop. It comes from the timing fit and from
`rate_stability()`, which measures how far the oscillator drifted during a run:
**186 to 1241 ppm** here, monotonically downward. Let the rig settle first.

The output is a different format and `capture.py` will refuse to read it, on
purpose — raw captures stay the one source of truth. Its header carries the
source's SHA-256 so a derived file that has drifted from its source is
detectable rather than merely plausible. Full analysis in
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md).

> `Capture.rate_error` is measured against the **nominal** model, whose error is
> cycle-count dependent — +0.84% at cc=100 and −0.99% at cc=400 on one unit, same
> supply. It is a configuration diagnostic, not a gain measurement, and
> multiplying a field by `1 + rate_error` is not a correction. `calibrate.py`
> does that against a measured reference instead.

A capture is analysable whether the run ended on `--duration`, on **Ctrl-C**, or
early — it is valid up to wherever it stopped, so the analysis tools can be
pointed at it either way.

## Capture format

A capture stores **only irreducible facts** — the chip's sample count, the host
clock, and the raw counts — behind a header of configuration. Chip time, elapsed
time, tesla and magnitude are all reconstructed on load by `capture.py`, so there
is exactly one source of truth for each.

```
# rm3100_capture: 1
# nominal_rate_hz: 282.53110196547465
# tmrc_nominal_hz: 600.0
# tmrc: 0x92
# cycle_count: 100
# tesla_per_count: 2.617115938236064e-08
# i2c_address: 0x23
# bus_speed_khz: 750
# revid: 0x22
# calibrated_period_s: 0.0035391156462585034
# note: bench, 3V0 LDO
sample_index,system_time_unix,x_raw,y_raw,z_raw,warning
0,1755930856.722866,-2765,767,378,
5,1755930856.867436,0,0,0,MISSED
13,1755930857.041244,0,0,0,MISSED AMBIGUOUS
21,1755930857.272108,-2761,769,377,AMBIGUOUS
```

`tesla_per_count` is written with `repr()` so it round-trips through float64
exactly, and is expressed per-count rather than the datasheet's LSB/µT so
conversion is a multiply: `tesla = count * tesla_per_count`. All times are unix
epoch seconds; there is no ISO-8601 anywhere in the data.

The `warning` column carries space-separated flags, empty when the row is fine,
and generalises to future flags. Because it already marks a row as having no
data, placeholders carry **zeros rather than blanks**, so `x/y/z` parse as
integers on every row without special-casing.

| flag | meaning |
|---|---|
| `MISSED` | placeholder row; the measurement was never read, so it has no data. The row exists to keep `sample_index` contiguous |
| `AMBIGUOUS` | the gap ending at this row was of uncertain length, so the index may have slipped from here on |

**The two are independent.** A gap measuring 1.35 periods rounds to one, so no
placeholder is written — yet it sits far enough from an integer to distrust, and
that case carries `AMBIGUOUS` on the *real* sample ending the gap, which keeps
its data. A placeholder inside an uncertain gap carries both.

The flag is a confidence measure, not a claim that 1.35 and 1.65 are equally
likely — 1.35 probably is one period and 1.65 probably two. Deciding which needs
neighbouring timestamps and assumptions and can still be wrong, which is exactly
why these rows are flagged rather than silently resolved.

`capture.py` interpolates `MISSED` rows on load so the uniform grid the spectra
depend on survives, and reports how many were substituted so it is never silent.

## How it works

**The chip samples on its own internal schedule**, unaffected by bus traffic
(§5.8.2 *"this can run in the background"*; §5.7.2 *"This will not affect the
measurement process"*). Two consequences drive the whole design:

- **Read jitter is not sample jitter.** The measurement grid stays uniform even
  when host reads are late, so `sample_index` is a grid coordinate.
- **A late read returns the *newer* measurement**, not a delayed one. An
  unnoticed miss therefore skips a grid point and silently compresses the time
  axis.

So a lost measurement is **recorded, not dropped**: a placeholder row keeps
`sample_index` contiguous and makes the gap explicit. The run continues and
reports the total at the end. Double-counting is structurally impossible —
`HSHAKE` DRC1=1 makes DRDY clear only on a results read, and `configure()`
verifies the readback rather than assuming it.

**The period is calibrated before recording starts.** Counting how many grid
points passed unseen needs the real period, and the TMRC table value is 6–9% out
on this unit — enough to insert the wrong number of placeholders and slip the
index. `calibrate_period()` takes a least-squares slope over one second of
loss-free samples, which pins it to ~0.07%, then keeps refining it from clean
intervals so it follows the oscillator's thermal drift. If no clean stretch can
be found the run aborts before recording anything: that is the honest signal that
the requested rate is not sustainable.

**Counting lost measurements uses two different signals**, because the two
questions have different best answers:

- *Did we lose any?* — the **DRDY bracket**, the span between the last poll
  showing DRDY clear and the poll showing it set. This is exact rather than
  heuristic: if DRDY reads clear at t꜀ then every earlier measurement has already
  been read, measurements complete one period apart, so a bracket narrower than a
  period can contain at most one completion — and DRDY going high proves it
  contained at least one.
- *How many?* — the **interval since the previous accepted sample**, rounded to
  whole periods. Every accepted read sits on a grid point, so that interval is
  nearly an exact multiple. The bracket cannot say, since it only reaches back to
  the last poll that saw DRDY clear and so discards where the grid is.

The read-to-read interval alone cannot make the first claim: measured here it
reaches 37 ms against a 28.8 ms period — a 29% overshoot from host stalls alone —
while the bracket stays under 10 ms. Thresholding the interval flags healthy
captures as lossy.

**Clock drift is compensated in post-processing, not baked into the file.** The
RC oscillator is *regular* but only accurate to ±7% (§5.2.1). `capture.py`
regresses the host clock on `sample_index` to recover the true period:

```
elapsed_nominal = N * dt_nominal    # uniform, but wrongly scaled
elapsed         = N * dt_true       # uniform and correctly scaled
```

Short-term regularity comes from the chip, long-term rate calibration from the
host. `characterize.py` uses `dt_true` for its frequency axis, since a 6% error
would displace every spectral feature by 6%. Over hours the oscillator drifts
enough that no single slope fits; `capture.py` reports that as `drift_limited`
and says to analyse shorter windows.

**Threading.** The sampling loop does I²C and nothing else, pushing raw counts
onto a bounded queue; a writer thread does unit conversion, CSV formatting,
flushing and the console display. The sampler releases the GIL inside each USB
transfer and must re-acquire it, so Python's default 5 ms switch interval becomes
the jitter floor — `sys.setswitchinterval(0.0005)` cut the worst bracket from
9.96 ms to 5.65 ms. The writer is also reniced out of the way, which needs no
privilege; `--high-priority` additionally raises the sampler, which does.

If the queue ever fills, the run **stops** rather than dropping a row: a missing
row would leave a hole in `sample_index`, and a truncated capture is worth more
than a longer one no tool can load.

## Configuration

**Cycle count is the rate knob, not TMRC.** Two ceilings compete and the slower
one wins (§5.2.1): the cycle count sets how *long* a measurement takes, TMRC sets
how *often* one is started.

| | sets | granularity |
|---|---|---|
| cycle count | `3 × (cc/90,000 + 68.7 µs)` per measurement | continuous |
| TMRC | how often a measurement starts | factor-of-two steps |

> The 90,000 counts/s is the specified figure and holds up; **the 68.7 µs
> overhead does not.** Solving the model against two cycle counts on this unit
> gives 40.6 µs at 3.0 V and 38.1 µs at 3.35 V, so the predicted cc100:cc400 rate
> ratio is 1.9% out. The overhead also moves with the supply, because the same
> oscillator times it — see below.

Leaving TMRC faster than the cycle-count ceiling makes the sensor free-run at
~100% duty and renders TMRC irrelevant. Setting it slower makes the sensor idle,
which costs sensitivity for nothing: noise after filtering scales as `1/√duty`,
and a measured 23% duty cost 1.43× the noise ASD against the same rate reached by
cycle count alone. So the default in every branch is the fastest TMRC, and
`--rate` derives the cycle count from there.

**Cycle count also sets resolution**, which is the real trade:

| cc | rate | Nyquist | nT/LSB | dither | duty | |
|---|---|---|---|---|---|---|
| 50 | 534 Hz | 267 Hz | 50.37 | 0.58 | 89.0% | spectrum, thin dither |
| **100** | **283 Hz** | **141 Hz** | **26.17** | **0.79** | **94.2%** | **default** |
| 200 | 145 Hz | 73 Hz | 13.35 | 1.10 | 97.0% | resolution, 60 Hz only |
| 400 | 74 Hz | 37 Hz | 6.74 | 1.54 | 98.5% | mains aliases, fs/4 artefact |

*dither* is the sensor's own noise in LSB. Below roughly 0.2 LSB the quantiser
stops being dithered and averaging no longer recovers sub-LSB resolution.

**Measured, not assumed:** at cc=100 the dither came out at 0.65–0.81 LSB and
averaging 1,024 samples reached **0.035–0.050 LSB** — a factor of 20 below the
step, within 1.5–2.1× of the ideal `1/√n`. The quantiser is not stalling.
`characterize.py` prints this for any capture; see
[Noise_Floor_Testing/NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md) §2.

**The default is cycle count 100 at 750 kHz**, which runs the sensor at its own
~283 Hz ceiling. It sits deliberately between the two things pulling in opposite
directions:

- **Against cc=200** it costs 1.5% in post-filter noise for 1.9× the spectrum
  (141 Hz of Nyquist against 73 Hz). Worth taking, because **aliased interference
  cannot be filtered out afterwards at any cycle count** — at 283 Hz both mains
  and its second harmonic are in band and can be notched.
- **Against cc=50** it gives up half the spectrum and buys 36% more dither
  margin. cc=50 is right when something above 141 Hz needs identifying; it is not
  the right default, because its dither margin is the thinnest here and the only
  one still unmeasured.

Fall back to cycle count 200 if `characterize.py`'s dither check shows the LSB
column flattening rather than continuing to fall.

**Prefer decimating a fast capture over sampling slowly.** Measured on a cc=100
capture decimated by 4 against a natively-recorded cc=400 one: decimation changes
a capture's own broadband floor by **+0.3 to +0.5%**, so the two are equivalent
for noise — and decimation is strictly better on everything else. The natively
slow capture folds 60 Hz irrecoverably to 13–17 Hz and carries sample-locked
lines at fs/4 and fs/2 that the decimated path does not have at all. `compare.py`
prints this comparison whenever two captures differ by an integer cycle-count
factor.

**Bus speed is independent of the rate** — it appears in neither mechanism. What
it sets is latency: how long a read takes, hence how tightly DRDY can be
timestamped and how much margin there is against a stall. Host cost is
`bus time + ~0.6 ms` of fixed USB round trip, so faster is simply better.

| bus | traffic/sample | share of a cc=100 period |
|---|---|---|
| 100 kHz | 1.500 ms | **42%** |
| 400 kHz | 0.375 ms | 11% |
| **750 kHz** | **0.200 ms** | **6%** |

Below ~0.46 Hz the 16-bit cycle-count register runs out and TMRC must set the
cadence; below cycle count 30 the manual warns of quantisation (§5.1). Both
bounds are enforced by `rm3100.plan()`.

### Configuration warnings

`rm3100.plan()` resolves a configuration, `logger.py` prints its derivation, then
checks it against six known traps. Each is a *silent* failure — the capture
completes, the numbers look plausible, and the defect only shows up afterwards.
So each is reported on stderr and **nothing is fixed automatically**: changing a
setting that was asked for would hide the problem behind a configuration change.

| Warning | Trigger | Why it matters |
|---|---|---|
| cycle count below the recommended 50 | `cc < RECOMMENDED_MIN_CYCLE_COUNT` | dither thins toward the ~0.2 LSB where averaging stops recovering sub-LSB resolution |
| TMRC governs and the sensor idles | TMRC-governed, idle > 20% | idle time buys nothing; measured 1.43× the ASD at 23% duty |
| rate differs from the one requested | \|error\| > 2% | the run silently uses the ceiling, and every derived figure moves with it |
| Nyquist below 60 Hz | rate < 120 Hz | mains folds onto signal and no later filter undoes it |
| cycle count past 400 | `cc > MAX_SPEC_CYCLE_COUNT` | Table 3-1 ends there, so the printed gain and noise are extrapolated |
| bus over half the period | traffic / period > 50% | names the speed that would fit |

The aliasing one is the easiest to walk into: `--rate 32` yields a clean-looking
2.9 nT/LSB capture with 60 Hz mains sitting at 4.0 Hz, indistinguishable from
signal. Sampling fast and decimating afterwards gives the same noise floor with
the line still visible.

## Diagnostics

`diagnose-comms.py` walks the chain — USB → bus → identity → registers → BIST →
live read — and stops at the first failure, so a fault points at a specific wire.
Otherwise, symptoms map onto causes:

| Symptom | Cause |
|---|---|
| `Cannot claim CH347 interface 2: Access denied` | udev rule missing or not yet applied to this node — replug the adapter, then run `./setup.sh`, which reports whether `uaccess` or group membership granted access and diagnoses whichever fell through. On Bazzite also try `sudo udevadm control --reload-rules && sudo udevadm trigger`. Do **not** reach for `sudo`: it works as a normal user, and masking a permissions problem with root only defers it |
| `No CH347 adapter found` | not plugged in, or not in Mode 1 |
| `Could not find N s of loss-free samples` | the host cannot sustain this rate — lower it (`--rate`, or a higher `--tmrc`) or raise `--bus-speed` |
| `N measurement(s) were lost` | host stalls during the run. Same fixes; the capture is still usable, with explicit gaps |
| `N gap(s) could not be counted confidently` | the index may have slipped — re-record before doing spectral work |
| `no capture header found` | a capture predating the header format — re-record it |
| `sample_index is not contiguous` | the file was truncated mid-row, damaged, or hand-edited |
| `HSHAKE did not take` | the I²C write is unreliable; try a lower `--bus-speed` |
| Adapter opens, `--scan-only` finds nothing | **I2CEN not tied high** (most likely), or SDA/SCL swapped, or no bus pull-ups |
| Found at 0x23, REVID `0x22`, registers fine, but DRDY never sets and results are all zero | **AVDD/VDD not powered** — confirm with BIST |
| Cycle-count read-back mismatch | bus integrity — try a lower speed |

**BIST is the definitive test for the analog side** (§5.6.1). Write `0x8F` to
BIST (STE=1, max timeout and periods), write `0x70` to POLL, wait, then read BIST
back: bits 4/5/6 are XOK/YOK/ZOK, and `1` means that axis's LR oscillator ran.
All zeros means the coils are not oscillating, pointing at AVDD or the REXT
timing resistor rather than anything on the I²C side.

A healthy total field magnitude is roughly **25–65 µT** (Earth's field). Near
zero, railed, or wildly out of range means the decode or the gain is wrong rather
than merely "data arrived".

## Files

| File | Role |
|---|---|
| `setup.sh` | udev rule, venv, dependencies, node permission check |
| `ch347.py` | CH347 I²C transport — USB only, knows nothing about the sensor |
| `rm3100.py` | RM3100 driver and configuration model — knows nothing about USB |
| `logger.py` | CLI: scan → identify → configure → calibrate → log, recording lost measurements as explicit placeholders |
| `capture.py` | the only capture reader — parses the header, rebuilds tesla and both time bases |
| `plot.py` | four-panel plot of a capture: X, Y, Z and the norm |
| `characterize.py` | noise floor: spectral density, Allan deviation, residuals, read latency |
| `sweep.py` | sweep target rates, reporting measured rate, LSB, noise, duty and bus use per point |
| `compare.py` | A/B captures — fractional noise, plus the checks that separate a gain change from a moved sensor |
| `calibrate.py` | convert a raw capture to nanotesla, applying a per-unit calibration if one is given; also holds the calibration model |
| `diagnose-comms.py` | walks USB → bus → identity → registers → BIST → live read, stopping at the first failure |
| `tests/` | pytest suite; needs no hardware |
| `Noise_Floor_Testing/` | captures, figures and [NOISE_FLOOR.md](Noise_Floor_Testing/NOISE_FLOOR.md) — the supply and rate-scaling analysis. Untracked: `.gitignore` excludes it |

The adapter/sensor split is deliberate: `ch347.py` is a general I²C master usable
with any device, and `rm3100.py` needs only a bus object exposing
`write(addr, data)` and `read(addr, count)` — optionally `write_read()`, which it
prefers when available.

## Implementation notes

Things that cost time to work out, recorded so they don't have to be again.

### CH347 access

The adapter presents three USB interfaces. Interfaces 0 and 1 are CDC-ACM and the
kernel binds them as `/dev/ttyACM0` (the UART). **Interface 2** is vendor-class,
carries I2C/SPI/JTAG with no kernel driver bound, and libusb can claim it
directly with nothing to detach. `ch347.py` deliberately never calls
`set_configuration()` — the device is already configured, and re-setting it would
disturb the CDC-ACM interfaces driving the UART.

There is no usable off-the-shelf option: no CH347 kernel driver exists (the
in-tree `spi-ch341` is for `1a86:5512`, a different chip), the PyPI `ch347`
package wraps a Windows DLL, and `ch347api` supports only HID mode (`55dc`), not
this vendor-bulk Mode 1. The framing follows the `aystarik/ch347-i2c-spi-gpio`
Linux driver, whose id-table entry
`USB_DEVICE_INTERFACE_NUMBER(0x1a86, 0x55db, 0x02)` matches this device exactly.

Wire format — bulk OUT `0x06`, bulk IN `0x86`, max 63 bytes per transfer:

| Purpose | Bytes out | Bytes back |
|---|---|---|
| Set speed | `AA 6<speed> 00` | 0 |
| Write n | `AA 74 (80\|n+1) (addr<<1) <data> 75 00` | n+1 |
| Read n | `AA 74 81 ((addr<<1)\|1) [C0\|(n-1)] C0 75 00` | n+1 |
| Write n then read m | `AA 74 (80\|n+1) (addr<<1) <data> 74 81 ((addr<<1)\|1) [C0\|(m-1)] C0 75 00` | n+2+m |
| Probe | `AA 74 81 (addr<<1) 75 00` | 1 |

**Every returned byte must be `1`** — that is the per-byte ACK. On a read, the
leading bytes are address ACKs and the rest is payload. `tests/test_ch347.py`
pins all five forms byte for byte.

### RM3100 quirks

- **Register reads use the plain address, not `|0x80`.** §5 describes the SPI
  convention of adding 0x80, but the I2C diagram in §5.8.4 writes `0x24`
  literally. Only 7 bits are decoded, so both work; the plain form is used here.
- **A register read uses a repeated START**, not the STOP-then-START the manual
  draws (§4.5.2, §5.8.4). The part accepts it — verified against REVID and the
  measurement registers — and it halves the USB round trips, which is what sets
  the sample-rate ceiling. `rm3100.read_reg()` falls back to the manual's form on
  a bus that cannot do it.
- **CMM = 0x79** for all three axes. Table 5-1 describes bit 3 as reserved-zero,
  but the manual's own examples (§5.7.2, §5.8.3) set it. The examples win.
- **HSHAKE is set to 0x0A** (`DRC0=0`, `DRC1=1`) during init. The 0x1B default has
  `DRC0=1`, meaning *any* register write clears DRDY — including the pointer write
  that reading STATUS itself requires, so polling could never observe DRDY set.
  With `DRC0=0`, DRDY clears only on a results read, which is what makes
  exactly-once sampling possible.
- **Gain** is `0.3671 × cycle_count + 1.5` LSB/µT, a linear fit to Table 3-1
  (50→20, 100→38, 200→75; reproduces all three within a count).
- **The rate model divisor is specified, not fitted.** Table 3-1 gives a 180 kHz
  circuit oscillation and §4.1 measures each cycle count in *both* bias
  directions, so one count costs two oscillations — hence 90,000 counts/s. The
  68.7 µs per-axis overhead has no specified value and was measured; it is why
  the naive `rate × cc` "constant" drifts from 84,429 at cc=100 to 89,191 at 1200.
- Measurements are three **24-bit big-endian two's-complement** values, read as
  9 bytes from 0x24 using the sensor's register auto-increment.

## Tests

```bash
./.venv/bin/python -m pytest            # whole suite, no hardware needed
./.venv/bin/python -m pytest -k logger  # one module
```

The suite fakes the hardware at three seams: `FakeUsbDevice` (libusb, so
`ch347.py`'s framing is itself under test), `FakeBus` (an I²C master with a
register map), and `FakeSensor` (a scripted DRDY timeline against a fake clock,
so a host stall of an exact size can be injected). See `tests/conftest.py`.

The miss-counting logic gets the most attention, since it is the part whose
failures are invisible downstream: a stall of 1.1 periods must lose nothing, 2.1
must record one placeholder, and 1.35 must flag the sample as `AMBIGUOUS` without
inserting anything.

## Status

- [x] udev rule, venv and dependency setup
- [x] CH347 vendor protocol — verified byte-for-byte against the kernel driver
- [x] I²C bus scan, sensor identified at 0x23, REVID `0x22`
- [x] Analog section verified — BIST reports `XOK=YOK=ZOK=1`
- [x] Exactly-once sampling, chip-grid time base, explicit gaps for lost samples
- [x] Verified on hardware: exact grid, −6.11% drift recovered, 0.017% run-to-run
- [x] Calibrated period, configuration warnings, cycle-count rate model
- [x] 3.0 V LDO built and compared — noise indistinguishable once normalised
- [x] Test suite covering everything that does not need the adapter
- [x] Interleaved supply A/B at two cycle counts — **scale factor quantified at
      −7.1% for +0.347 V**, rate at +4.94%
- [x] Dither margin at cc=100 checked against real data — averaging reaches
      0.035 LSB, so the quantiser is not stalling
- [x] Filter-and-decimate shown equivalent to sampling slowly (+0.5% on the
      broadband floor), and better on aliasing and artefacts
- [ ] Re-test the LDO with the sensor **clamped**, to separate gain from movement
      — every pair in the A/B still shows 2.8–23° of rotation
- [ ] Identify the **fs/4 artefact** that appears at cc=400 and not at cc=100;
      a cc=200 run at both supplies would say whether it scales with cycle count
- [x] Per-unit timing and gain calibration — `calibrate.py`, written by
      `sweep.py --calibration-out`, nothing corrected by default
- [ ] Measure a calibration on the bench and check the oscillator against a
      third rail voltage, which would separate V from V² in the gain scaling
- [ ] Deal with 60 Hz coupling at source
- [ ] Reliability testing over extended runs on bare metal

Captures written before the current header format are unreadable and must be
re-recorded — `capture.py` says so explicitly rather than guessing. Figures
quoted in [NOTES.md](NOTES.md) came from those files and stand as historical
measurements only.

A proper Python API is planned; these modules are the initial-communication
milestone, deliberately kept simple.