6f85a6f1ed
Pipeline (process_gps.py): - New standard solution ppk_vrs: a virtual reference station synthesized at the capture position from the GCGC master station the CORS stage already fetched (zero baseline). Gated on GCGC availability; --no-vrs to disable. Engine validated in experiments/ (gates G1/G2a) and benchmarked equivalent to Trimble Pivot's commercial VRS on ground truth. - Track output now selects among PPK-family solutions by measured dispersion, not nominal baseline (protects against poor network-edge VRS data). - Warm-up exclusion: first 5 minutes (--warmup-min) removed from all quality metrics and figures; prominent note in both reports; data outputs unchanged. - Professional report language: abbreviations expanded on first use, solution methods defined, no shorthand in tables or figure labels; fig5 reframed as "self-reported precision (uncalibrated) vs measured dispersion" with the optimism factor annotated. - Local-base robustness: refuse to combine base files at different positions; RTCM logs and Data Shop RINEX accepted for the parallel GCGC method. Streaming (stream_gps.py): - Credentials via gitignored gcgc.env (template gcgc.env.example), env vars take precedence; RTCM correction stream recorded to stream_<ts>.rtcm3 for zero-baseline post-processing; live GGA uplink and sea dynamic model remain the buoy defaults; warm-up excluded from session statistics. Experiments (new): - vbs_synth.py: geometric virtual-base synthesis engine (RINEX 2.11 patcher, SP3 orbits, light-time + Earth-rotation per position, clock-robust). - vbs_iono.py: carrier-leveled slant-ionosphere estimation + station-network interpolation with leave-one-out validation (median 7 cm; 1.2 cm / 10 km growth; DCBs as daily constants via median polish). - vbs_compare.py: observation-domain comparison of commercial VRS files vs synthesized bases (ambiguity-detrended correction content). - FINDINGS.md: gate results, commercial benchmark, offshore analysis, and promotion rationale. Docs: README validation-results section explaining why the synthesized VRS (kinematic) is the preferred solution for buoy deployments; VRS orders and experiment data moved under gitignored experiments/data/.
194 lines
11 KiB
Markdown
194 lines
11 KiB
Markdown
# VBS Exploration Findings — Session 1 (2026-07-27)
|
||
|
||
Exploration of DIY virtual-base-station synthesis from GCGC/NGS multi-station
|
||
data, targeting offshore (extrapolation) PPK for buoy deployments. Full design
|
||
+ gate definitions: see the approved plan; scripts: `vbs_synth.py`,
|
||
`vbs_iono.py`. Data: NGS CORS 30 s dailies, DOY 204 (2026-07-23) — MSIN, MARY,
|
||
SBCH, COVG, MSEV (+ ALDI/ALFO/AL90/AL92/ALMJ downloaded for geometry D, unused
|
||
so far); ESA rapid SP3 (GPS+GLONASS — Galileo pending a multi-GNSS SP3 source,
|
||
e.g. ESA finals ~2 weeks post-date); NGS brdc RINEX 2 nav (demo5 rnx2rtkp did
|
||
NOT parse BKG's RINEX 3.05 mixed nav — use NGS brdc `.26n/.26g`).
|
||
|
||
## Gate G1 — geometric synthesis engine: **GREEN**
|
||
|
||
| Check | Result |
|
||
|---|---|
|
||
| G1a identity (zero displacement) | **PASS** — 44,884 surviving records bit-identical (max diff 0.000, flags preserved) |
|
||
| G1b round-trip MSIN→MARY site→back | **PASS** — max diff 0.001 (= 1 LSB of RINEX quantization) |
|
||
| G1c clock perturbation | **PASS at physical magnitude** (see below) |
|
||
| G1d A→B truth (MARY as rover, 43.7 km) | **PASS** — VBS-at-MARY base reproduces the direct MSIN-base solution (mean 3D diff 13 cm = float noise); truth errors statistically identical (horiz RMS 0.634 vs 0.639 m; mean N/E offset actually slightly smaller for VBS) |
|
||
|
||
G1d detail (4 h window 16–20 UT, GPS+GLO, 30 s, broadcast nav, kinematic):
|
||
direct fix rate 37.4% with **69 wrong fixes** (Q=1 & >10 cm truth error); VBS
|
||
43.7% with 31 wrong fixes. Wrong fixes are present in BOTH — confirming the
|
||
plan's central honesty rule: never score on fix rate; score truth error under
|
||
Q=1. 44 km daytime solar-max AR is marginal, as expected.
|
||
|
||
### The clock findings (validates the timing concern raised in review)
|
||
|
||
- A **1 ms** injected clock offset broke the solution (0.63 m shifts, AR lost)
|
||
— but diagnosis showed the *test* was unphysical, not the engine: shifting
|
||
observables without shifting the sampling instant creates an inconsistent
|
||
receiver. RTKLIB derives the base clock from pseudoranges and time-shifts
|
||
satellite positions accordingly — so timestamp↔pseudorange **self-consistency
|
||
in synthesized files is mandatory** (our engine preserves the master's real,
|
||
self-consistent clock, hence G1d passing).
|
||
- At the physically representative **1 µs** (steered CORS clocks): both-float
|
||
epochs agree to **max 4.7 mm / mean 0.8 mm** → the engine is clock-robust.
|
||
All larger deviations (≤12 cm) came from 18/481 epochs where a *borderline AR
|
||
validation decision flipped* — chaotic threshold sensitivity inherent to
|
||
marginal AR, present under any tiny perturbation, not a synthesis defect.
|
||
|
||
## Gate G2a — iono LIM cross-validation: **GREEN (GPS-only)**
|
||
|
||
Carrier-leveled geometry-free slant iono (arc-leveled to code), 5-min bins,
|
||
20° mask, per-satellite planar fit (LIM) across 4 stations, leave-one-out:
|
||
|
||
| Held-out | Centroid dist | median\|r\| | 95%\|r\| |
|
||
|---|---|---|---|
|
||
| MSIN | 18 km | 0.050 m | 0.158 m |
|
||
| MARY | 52 km | 0.050 m | 0.124 m |
|
||
| SBCH | 65 km | 0.057 m | 0.185 m |
|
||
| COVG | 38 km | 0.100 m | 0.267 m |
|
||
| MSEV | **135 km (true extrapolation)** | **0.191 m** | 0.551 m |
|
||
|
||
- **Overall median 0.070 m ≤ 0.10 m gate → PASS.**
|
||
- **Residual-growth slope: 1.2 cm per 10 km** — matches the literature band
|
||
(1–2 cm/10 km) used in the design review. Offshore implication: ~10–15 cm
|
||
slant-iono prediction error at 60 km beyond the network edge — supportive of
|
||
dm-level (not cm-level) VBS improvement targets for extrapolated points.
|
||
|
||
### Two instructive failures on the way (both diagnosed + fixed)
|
||
|
||
1. **v1 estimator failed the gate by 10x** (median 0.7–1.4 m; COVG 4.6–6.9 m).
|
||
Cause: per-bin free per-station bias parameters are near-degenerate with a
|
||
satellite-common gradient in a 4-station fit — extrapolated planes explode.
|
||
Fix: estimate station biases (DCBs) as **daily constants** via median polish
|
||
(hardware DCBs are stable), then fit per-bin planes with biases fixed →
|
||
residuals collapsed 20x. Lesson recorded for Phase 2b: DCB handling is
|
||
*the* conditioning issue in small-network LIM.
|
||
2. **P1/C1 column trap**: MSEV logs GPS code in C1 with a blank P1 column —
|
||
column-level fallback silently produced a GLONASS-only station. Value-level
|
||
P1→C1 / P2→C2 fallback required. (GLONASS remains excluded from the iono
|
||
fit for now: per-slot inter-channel code biases need per-(station,slot)
|
||
bias terms — queued for Phase 2b if GPS-only corrections prove insufficient.)
|
||
|
||
## Pivot VRS benchmark (2026-07-27, orders V304-V307) — **the G0 question answered**
|
||
|
||
GCGC's portal DOES offer VRS orders, and **accepted both offshore points (25 km
|
||
and 60 km beyond the network edge) without complaint** — 1 s files, zero missing
|
||
epochs, positions declared exactly as requested (NAD83(2011); note +1.4 m vs
|
||
ITRF ellipsoidal height in this region). Files in `experiments/data/orders/` (V304 = capture
|
||
site 50 min; V305 = MARY site 4 h; V306/V307 = offshore probes 4 h).
|
||
|
||
**MARY truth benchmark** (rover = MARY 30 s, 16-20 UT, GPS+GLO, broadcast nav;
|
||
truth = MARY's published coordinates, frame-matched per leg):
|
||
|
||
| Base | Fix rate | Wrong fixes | Truth horiz RMS | Fixed-only RMS / CEP95 |
|
||
|---|---|---|---|---|
|
||
| Pivot VRS at MARY (0 km) | 41.4% | 45 | 0.548 m | **0.078 / 0.129 m** |
|
||
| DIY VBS at MARY (0 km, geometric-only) | 43.7% | 31 | 0.634 m | **0.075 / 0.149 m** |
|
||
| Direct MSIN (44 km) | 37.4% | 69 | 0.639 m | 0.136 / 0.298 m |
|
||
|
||
Findings:
|
||
1. **Commercial Pivot VRS ~= DIY geometric-only VBS at this site/day** — the
|
||
network's atmospheric corrections added no measurable advantage over pure
|
||
geometric displacement (single quiet-ish day, GPS+GLO, 30 s caveats apply).
|
||
Both zero-baseline methods beat the 44 km direct base *when fixed* (7.5-7.8
|
||
cm vs 13.6 cm) and roughly halve wrong fixes.
|
||
2. **Wrong-fix rates are high for ALL methods** (~20-35% of Q=1 epochs) —
|
||
processing-config improvements (fix-and-hold, mask, L5) are currently a
|
||
bigger lever than base choice; reinforces truth-error-under-fix scoring.
|
||
3. **Capture-site leg (V304 via the production pipeline)**: ppk_gcgc ran
|
||
automatically (base 0.0 km) but stayed float with ratio ~1.1 and *worse*
|
||
scatter than the MSEV solution (CEP50 1.10 vs 0.44 m), plus a ~3.5 m
|
||
unexplained height offset beyond the ~1.4 m datum difference — float-bias
|
||
behavior consistent with the u-blox rover's GAL/BDS signals being unusable
|
||
against a G+R-only 2.11 base plus VRS correction noise; needs a dedicated
|
||
look (multi-frequency conf, `-f 3`, RINEX 3.04 VRS re-order at the site).
|
||
4. Offshore V306/V307 files are in hand for the Phase 2b/3 obs-domain
|
||
comparison (Pivot's extrapolated corrections vs our LIM extrapolation).
|
||
|
||
## Obs-domain analysis: how much correction does Pivot actually embed? (vbs_compare.py)
|
||
|
||
DIY geometric VBS synthesized at each Pivot file's exact declared position
|
||
(same NAD83 frame/master), observations differenced satellite-by-satellite,
|
||
DD'd against the highest satellite, per-satellite ambiguity constants removed.
|
||
What remains = network correction content relative to pure geometry (+ master
|
||
noise; 95% tails include uncleaned re-levelings):
|
||
|
||
| VRS point | Displacement | L1 DD variation median / 95% | P1 code DD median |
|
||
|---|---|---|---|
|
||
| V304 capture site (edge) | 34 km | **0.026 / 0.163 m** | 0.278 m |
|
||
| V305 MARY (in-network) | 29 km | **0.064 / 2.127 m** | 0.369 m |
|
||
| V306 25 km offshore | 75 km | **0.063 / 0.914 m** | 0.446 m |
|
||
| V307 60 km offshore | 92 km | **0.065 / 1.020 m** | 0.517 m |
|
||
|
||
**Three independent measurements now agree**: (1) LOOCV says LIM-interpolable
|
||
iono differences are 5-10 cm median in-network, growing 1.2 cm/10 km; (2) the
|
||
MARY truth benchmark says that correction content is too small to change
|
||
positioning outcomes on this day (Pivot ~= DIY); (3) the obs-domain analysis
|
||
says Pivot embeds a median of only **~3-7 cm of correction beyond pure
|
||
geometry, roughly flat out to 92 km displacement** (code DD grows mildly with
|
||
distance, dominated by master code noise). Consistent conclusion for buoys:
|
||
at these distances/conditions, a geometric virtual base + our own LIM layer is
|
||
competitive with the commercial product; the decisive factors are processing
|
||
config (AR strategy, multi-frequency) and disturbed-day behavior (untested).
|
||
|
||
Capture-site V304 anomaly **RESOLVED (2026-07-27, promotion session)**: the
|
||
production ppk_vrs stage (our engine, same MSEV master, same zero baseline,
|
||
same rover) is clean - matches direct-MSEV to 3 cm and slightly beats its
|
||
scatter (CEP50 0.42 vs 0.44 m). The ~5 m bias + 3x noise is therefore **in
|
||
Pivot's V304 file itself** (network-edge synthesis at Hattiesburg where MSHT
|
||
is decommissioned), not in the rover pairing. Method note: the obs-domain
|
||
DD comparison could not see this because per-satellite ambiguity detrending
|
||
also absorbs constant biases - it measures time-varying content only. The
|
||
RINEX 3.04 re-order of the capture-site VRS is now optional curiosity, not a
|
||
blocker. Fix-and-hold truth test at MARY: +10% fix rate on the Pivot leg only,
|
||
wrong-fix counts UNCHANGED on all legs - wrong fixes are a validation-threshold
|
||
problem (30 s epochs, ratio 3.0), not an AR-strategy problem; the next real
|
||
levers are multi-frequency processing and stricter/partial validation.
|
||
|
||
## Promotion (2026-07-27, user decision)
|
||
|
||
The DIY VRS is promoted into `process_gps.py` as the **standard GCGC-provided
|
||
solution `ppk_vrs`**: auto-synthesized at the capture position from the NGS
|
||
stage's master whenever that master is a GCGC station (agency header / MS
|
||
station check), ESA orbits (FIN>RAP, ~1 day lag, "pending" until published),
|
||
frame-consistent ITRF, zero nominal baseline. Reported alongside all other
|
||
solutions with its own color/validation; `--no-vrs` disables. Track output now
|
||
selects among PPK-family solutions by empirical scatter, protecting against
|
||
poor network-edge VRS data (exactly the V304 case). Caveats carried with the
|
||
promotion: single-day validation, GPS+GLO only until multi-GNSS orbits are
|
||
available for the day (Galileo joins via ESA finals), disturbed-iono behavior
|
||
untested (Phase 3).
|
||
|
||
## Status vs plan
|
||
|
||
- G0: partially done — station set + data availability verified (NGS path);
|
||
**USER ACTION open: check the GCGC portal (rtn.usm.edu) "new order" screen
|
||
for a "Virtual Reference Station" order type**, and whether it permits
|
||
offshore points; GCGC Data Shop 1 s downloads not yet needed but will be for
|
||
Phase 3's disturbed-day / 1 Hz legs.
|
||
- G1: **GREEN** (all four checks).
|
||
- G2a: **GREEN** (GPS-only, one quiet day). Pending for robustness: ≥2 more
|
||
days incl. a disturbed day (Phase 3 requirement, not a G2a blocker).
|
||
- Next (Phase 2b): apply LIM corrections in `vbs_synth` (`--iono` hook exists
|
||
in design): sign-correct code/phase application, (f_L1/f)² scaling,
|
||
phase-continuity audit (<λ/4 steps), then geometry B/C/D truth runs with the
|
||
wrong-fix-rate scoring, day/night splits, and the 0–60 km offshore sweep.
|
||
|
||
## Reproduction
|
||
|
||
```bash
|
||
# G1 (identity/round-trip/clock/truth) — see session commands; engine:
|
||
venv/bin/python experiments/vbs_synth.py experiments/data/204/msin2040.26o \
|
||
--xyz <MARY_XYZ> --master-xyz <MSIN_XYZ> \
|
||
--sp3 experiments/data/204/ESA0OPSRAP_20262040000_01D_05M_ORB.SP3 \
|
||
--nav experiments/data/204/BRDC00WRD_R_20262040000_01D_MN.rnx \
|
||
-o experiments/data/204/vbs_at_mary.obs
|
||
# G2a:
|
||
venv/bin/python experiments/vbs_iono.py --stations msin:... mary:... sbch:... \
|
||
covg:... msev:... --dir experiments/data/204 --sp3 ... --nav ... --gps-only
|
||
```
|
||
(ITRF2020 station coordinates epoch-propagated via `process_gps.cors_station_xyz`.)
|