Initial commit of fully working GPS processing pipeline. See README for more information.
This commit is contained in:
=
commit
65a4836608
8 changed files with 2620 additions and 0 deletions
18
.gitignore
vendored
Normal file
18
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# Python environment
|
||||
venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# Bootstrapped tooling: RTKLIB build, ANTEX models, CORS catalog cache
|
||||
# (fully reproducible with setup.sh)
|
||||
tools/
|
||||
|
||||
# Pipeline outputs (regenerate with process_gps.py)
|
||||
processed/
|
||||
|
||||
# Live streaming session outputs (stream_gps.py)
|
||||
stream_*.csv
|
||||
|
||||
# Raw GPS capture data
|
||||
*.ubx
|
||||
*.uc2x
|
||||
193
README.md
Normal file
193
README.md
Normal file
|
|
@ -0,0 +1,193 @@
|
|||
# u-blox GPS Module Data Procesing (u-blox ZED-X20P)
|
||||
|
||||
Tooling for evaluating u-blox ZED-X20P all-band GNSS modules, targeting **buoy
|
||||
deployments**. Converts raw u-center capture files (`.ubx`) into corrected
|
||||
positions and produces a full diagnostics report, and supports live streaming
|
||||
with real-time corrections.
|
||||
|
||||
Two operating modes:
|
||||
|
||||
| Mode | Script | Accuracy | Correction source |
|
||||
|---|---|---|---|
|
||||
| Post-processing | `process_gps.py` | dm-cm (PPK) | NGS CORS base data, auto-fetched (free, open) |
|
||||
| Real-time streaming | `stream_gps.py` | 1-3 cm (RTK fix) | GCGC RTN NTRIP (free registration) or Galileo HAS (FW ≥ HPG 2.10) |
|
||||
|
||||
Both consume/produce standard formats (RINEX 3, `.pos`, CSV, KML) via RTKLIB
|
||||
(demo5 fork) and widely used open-source Python libraries (pyubx2, pygnssutils,
|
||||
numpy, pandas, matplotlib).
|
||||
|
||||
## Motion policy (important)
|
||||
|
||||
**Processing never assumes the antenna was stationary.** Buoys drift slowly and
|
||||
drift within a few metres is statistically indistinguishable from measurement
|
||||
noise. Everything defaults to kinematic; pass `--static` to `process_gps.py`
|
||||
only when the antenna is independently known to have been fixed (e.g. bench/roof test).
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
bash setup.sh # one-time environment bootstrap
|
||||
venv/bin/python process_gps.py <capture.ubx> # process a capture
|
||||
# results in processed/<capture-stem>/ (report.html is the human summary)
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
Linux with `python3` (3.10+), `gcc`/`make`, `git`, `curl`, and network access
|
||||
for `setup.sh` downloads. No sudo required and everything installs into the
|
||||
project directory (`venv/`, `tools/`).
|
||||
|
||||
---
|
||||
|
||||
## setup.sh
|
||||
|
||||
Environment bootstrap (safe to re-run; each step skips if already satisfied):
|
||||
|
||||
1. **Python venv** (`venv/`): pyubx2, pygnssutils, pyserial, numpy, pandas,
|
||||
pyproj, matplotlib.
|
||||
2. **RTKLIB demo5 v2.5.1** built from source into `tools/bin/`
|
||||
(`convbin`, `rnx2rtkp`, `pos2kml`). The demo5 fork is required: stock
|
||||
RTKLIB 2.4.3 silently drops all BeiDou and GPS L5 / Galileo E5 observations
|
||||
from this receiver (~30% of the measurements). If the build fails, setup
|
||||
falls back to the stock apt package and creates a `tools/STOCK_RTKLIB`
|
||||
marker — `process_gps.py` then adds a prominent data-loss warning to every
|
||||
report until a demo5 build succeeds.
|
||||
3. **Antenna calibration models** into `tools/antex/`: `igs20.atx` (IGS, covers
|
||||
CORS base-station antennas) and `ngs20.atx` (NGS absolute calibrations,
|
||||
covers the rover antenna). The rover antenna is resolved automatically and
|
||||
written to `tools/antenna.json` (see *Antenna configuration* below).
|
||||
4. **CRX2RNX** (Hatanaka RINEX decompressor) — fallback for CORS stations that
|
||||
only publish compressed `.d` observation files.
|
||||
5. **NGS CORS station catalog** cache (used for nearest-base auto-selection).
|
||||
6. Self-check summary of every tool and the resolved antenna.
|
||||
|
||||
## process_gps.py
|
||||
|
||||
```
|
||||
venv/bin/python process_gps.py <capture.ubx> [options]
|
||||
```
|
||||
|
||||
Stages (idempotent — outputs are cached, so re-running later only fills in
|
||||
what was previously unavailable, e.g. CORS/IGS data that had not been
|
||||
published yet):
|
||||
|
||||
1. **inventory** — parses the `.ubx` (RXM-RAWX/SFRBX raw observations,
|
||||
NAV-PVT/SAT/SIG, MON-RF) and the `.uc2x` sidecar's session metadata.
|
||||
A capture without raw observations (e.g. u-center debug-mode logs) gets a
|
||||
diagnostics-only report and a clear notice.
|
||||
2. **rinex** — RINEX 3.04 obs+nav via demo5 `convbin`; verifies epoch count and
|
||||
BDS/L5 presence; extracts broadcast ionosphere parameters.
|
||||
3. **single** — standalone single-point solution (~1 m, works offline).
|
||||
4. **ppk** — differential post-processing against the nearest operational NGS
|
||||
CORS station (auto-selected by distance from the capture's own mean
|
||||
position; MSEV at 34 km for the Hattiesburg test site). Hourly base files
|
||||
are fetched automatically (~1 h publication lag; if not yet published the
|
||||
stage reports "re-run later" and everything else proceeds). Base antenna
|
||||
type is read from the base RINEX header; base coordinates are ITRF2020,
|
||||
velocity-propagated to the capture epoch.
|
||||
5. **ppp** (`--ppp`) — precise point positioning against IGS rapid/final
|
||||
orbit+clock products (anonymous download from BKG/ESA, ~1 day lag).
|
||||
Kinematic PPP by default per the motion policy; note RTKLIB's PPP-kinematic
|
||||
engine is weak and may yield no solution — PPK is the reference method.
|
||||
6. **track** — `track.csv` (UTC + decimal-degree lat/lon + per-epoch quality
|
||||
and 1-σ sd columns + onboard comparison columns) and `track.kml`.
|
||||
7. **diagnostics** — `diagnostics.json`, `report.md`, `report.html`
|
||||
(self-contained, figures embedded) covering: constellation metrics (per-band
|
||||
C/N0, cycle slips, residuals), ionosphere (broadcast Klobuchar + measured
|
||||
dual-frequency slant delay/TECU), positional error metrics (RMS/R50/R95
|
||||
dispersion, formal σ, inter-solution offsets), **validation of every
|
||||
computed solution against the receiver's own onboard NAV-PVT positions**
|
||||
(epoch-matched, PASS/FAIL), timing (epoch jitter, tAcc, leap seconds), and
|
||||
RF health (jamming indicator, AGC, antenna supervisor state).
|
||||
|
||||
### Options
|
||||
|
||||
| Flag | Meaning |
|
||||
|---|---|
|
||||
| `--interval N` | `track.csv` output interval in seconds (default 1 = native rate) |
|
||||
| `--static` | antenna verifiably stationary: enables static PPK/PPP solutions and static error framing. **Never for buoys.** |
|
||||
| `--no-ppk` | skip base-station download/processing |
|
||||
| `--ppp` | additionally run PPP against IGS precise products |
|
||||
| `--base SSSS` | force a specific CORS station id (e.g. `MSEV`) |
|
||||
| `--max-base-km KM` | base-station search radius (default 150) |
|
||||
| `--force-stage S` | delete and re-run one stage (`rinex`,`single`,`ppk`,`ppp`,`track`) |
|
||||
|
||||
## Antenna configuration
|
||||
|
||||
Receiver-antenna phase-center corrections (PCO/PCV) matter at the cm level and
|
||||
are applied during PPK/PPP via ANTEX calibration files.
|
||||
|
||||
- **Automatic**: `setup.sh` downloads `ngs20.atx`/`igs20.atx` and searches for
|
||||
the project's antenna (ArduSimple calibrated quad-band, NGS calibration
|
||||
AS-ANT3BCAL01). The resolved 20-character ANTEX type string is written to
|
||||
`tools/antenna.json`:
|
||||
|
||||
```json
|
||||
{ "antex_name": "AS-ANT3BCAL NONE", "source": "ngs20.atx" }
|
||||
```
|
||||
|
||||
- **Different antenna**: edit `antex_name` in `tools/antenna.json` to the exact
|
||||
ANTEX type string of your antenna (padding/spacing matters, copy it verbatim
|
||||
from the `TYPE / SERIAL NO` line of an antenna block in
|
||||
`tools/antex/ngs20.atx` or `igs20.atx`). Browse available calibrations at
|
||||
https://geodesy.noaa.gov/ANTCAL/ . If your antenna has an NGS calibration but
|
||||
a different model string, also update the `ANT3B` search pattern in `setup.sh`
|
||||
so future setups re-resolve it automatically.
|
||||
- **No calibration available**: set `antex_name` to `null` (or leave it
|
||||
unresolved) — processing continues without rover PCO/PCV and the report
|
||||
carries a warning. Expect a few cm of systematic error, mostly in height.
|
||||
- **Base antenna** needs no configuration: its type is read from the CORS RINEX
|
||||
header and looked up in the same ANTEX files.
|
||||
- **Antenna height**: assumed 0 (positions refer to the antenna reference
|
||||
point). For a surveyed monument, subtract your ARP offset downstream.
|
||||
|
||||
## Other important settings and inputs
|
||||
|
||||
**Capture requirements.** Input `.ubx` files must contain raw observations:
|
||||
UBX-RXM-RAWX + UBX-RXM-SFRBX enabled (plus NAV-PVT; NAV-SAT/NAV-SIG/MON-RF
|
||||
enrich diagnostics). In u-center, do **not** record with the "debug messages"
|
||||
option — it floods the log with undocumented TRK/TUN/DBG/SEC messages and, in
|
||||
past sessions here, raw output was configured in flash but not active in RAM.
|
||||
The optional `.uc2x` sidecar (u-center 2 index) is only used for its session
|
||||
metadata header.
|
||||
|
||||
**Real-time credentials (`stream_gps.py`).** Free registration at
|
||||
http://rtn.usm.edu/RegisterAccount.aspx (Mississippi GCGC RTN), then:
|
||||
|
||||
```bash
|
||||
export GCGC_USER=<username> GCGC_PASS=<password>
|
||||
venv/bin/python stream_gps.py --port /dev/ttyUSB0 # or COM7 on Windows
|
||||
```
|
||||
|
||||
Key streaming defaults (buoy-oriented): receiver dynamic-platform model `sea`
|
||||
(`--dynmodel` to override), **live** NMEA GGA uplink so the network-RTK virtual
|
||||
reference follows platform drift (`--gga-fixed` for bench tests), raw
|
||||
RXM-RAWX/SFRBX always logged so every session is post-processable, and
|
||||
`--replay <file.ubx>` offline test mode. Galileo HAS (~20 cm, no internet)
|
||||
activates automatically as fallback when firmware ≥ HPG 2.10 is detected.
|
||||
|
||||
**Datums.** PPK output is in the CORS base frame (ITRF2020, current epoch);
|
||||
GCGC real-time corrections are NAD83(2011) epoch 2010.0 — a constant ~1.5 m
|
||||
offset from ITRF/WGS84 in CONUS. Each CSV records its frame in the header;
|
||||
comparisons in the report account for the reference used. Onboard receiver
|
||||
positions are WGS84-aligned.
|
||||
|
||||
**Data-source timing.** CORS hourly files: ~1 h lag, expire after ~2 days
|
||||
(daily files remain, decimated to 30 s after 30 days). IGS rapid products:
|
||||
~1 day lag. The pipeline caches everything it fetches under
|
||||
`processed/<stem>/` and tells you when a re-run will find more data.
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
setup.sh environment bootstrap (run first)
|
||||
process_gps.py post-processing pipeline (Mode 2)
|
||||
stream_gps.py real-time streaming client (Mode 1)
|
||||
run.sh convenience wrapper
|
||||
notes/ manual workflow notes + combine_rinex.sh helper
|
||||
tools/ built binaries, ANTEX models, caches (generated)
|
||||
venv/ Python environment (generated)
|
||||
processed/<capture>/ per-capture outputs: RINEX, .pos solutions,
|
||||
track.csv, track.kml, figures/, diagnostics.json,
|
||||
report.md, report.html (generated)
|
||||
```
|
||||
42
notes/RTK Processing Notes.txt
Executable file
42
notes/RTK Processing Notes.txt
Executable file
|
|
@ -0,0 +1,42 @@
|
|||
### THESE ARE NOTES FROM PREVIOUS WORK, KEPT FOR REFERENCE. NONE OF THESE STEPS ARE REQUIRED TO USE THE GPS DATA PROCESSING SCRIPTS IN THE PARENT DIRECTORY.
|
||||
|
||||
## Download the CRX2RNX tool precompiled binary
|
||||
|
||||
# Download the pre-compiled Linux binary
|
||||
cd /tmp
|
||||
wget https://terras.gsi.go.jp/ja/crx2rnx/RNXCMP_4.2.0_Linux_gcc_64bit.tar.gz
|
||||
|
||||
# Extract
|
||||
tar -xzf RNXCMP_4.2.0_Linux_gcc_64bit.tar.gz
|
||||
|
||||
# Install to system path
|
||||
sudo cp RNXCMP_4.2.0_Linux_gcc_64bit/CRX2RNX /usr/local/bin/
|
||||
|
||||
# Make executable
|
||||
sudo chmod +x /usr/local/bin/CRX2RNX
|
||||
|
||||
# Create lowercase symlink (rnx2rtkp looks for lowercase)
|
||||
sudo ln -s /usr/local/bin/CRX2RNX /usr/local/bin/crx2rnx
|
||||
|
||||
# Test
|
||||
crx2rnx -h
|
||||
|
||||
|
||||
## Download CORS corrections from here: https://www.ngs.noaa.gov/CORS/
|
||||
|
||||
## If base station CORS files are compressed (.25d format), decompress them
|
||||
ls msin*.25d | xargs -n 1 -P 4 crx2rnx
|
||||
|
||||
## Combine base station observations into one file
|
||||
./combine_rinex.sh msin_combined.obs msin*.25o
|
||||
|
||||
## Perform corrections on a .obs file using base station files as reference
|
||||
cd CORS\ STATION\ MSIN\ NOV\ 2025/
|
||||
|
||||
rnx2rtkp \ -t -u \
|
||||
-o ../../A/DATA/result.pos \
|
||||
../../A/DATA/20251120_033544.006_UTC_GPS_A_combined.obs \
|
||||
msin_combined.obs \
|
||||
../../A/DATA/20251120_033544.006_UTC_GPS_A_combined.nav
|
||||
|
||||
## The result.pos file now contains PPK corrected GPS data in LLH format
|
||||
141
notes/combine_rinex.sh
Executable file
141
notes/combine_rinex.sh
Executable file
|
|
@ -0,0 +1,141 @@
|
|||
#!/bin/bash
|
||||
# combine_rinex.sh - Combine RINEX observation files (supports both 2.x and 3.x)
|
||||
|
||||
if [ $# -lt 2 ]; then
|
||||
echo "Usage: $0 output.obs input_files..."
|
||||
echo "Example: $0 combined.obs msin*.25o"
|
||||
echo "Example: $0 combined.obs msin3240.25o msin3250.25o msin3260.25o"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
output="$1"
|
||||
shift
|
||||
|
||||
# Sort files chronologically (RINEX naming is sortable)
|
||||
files=$(printf '%s\n' "$@" | sort)
|
||||
|
||||
if [ -z "$files" ]; then
|
||||
echo "Error: No input files specified"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get first file
|
||||
first_file=$(echo "$files" | head -1)
|
||||
|
||||
# Verify all files exist
|
||||
echo "Verifying input files..."
|
||||
for file in $files; do
|
||||
if [ ! -f "$file" ]; then
|
||||
echo "Error: File not found: $file"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
file_count=$(echo "$files" | wc -w)
|
||||
echo "Found $file_count files to combine"
|
||||
echo ""
|
||||
|
||||
# Get header from first file
|
||||
header_lines=$(grep -n "END OF HEADER" "$first_file" | cut -d: -f1)
|
||||
|
||||
if [ -z "$header_lines" ]; then
|
||||
echo "Error: No RINEX header found in $first_file"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Detect RINEX version from first line
|
||||
rinex_version=$(head -1 "$first_file" | awk '{print substr($1,1,1)}')
|
||||
|
||||
if [ "$rinex_version" = "3" ]; then
|
||||
echo "Detected: RINEX 3.x format"
|
||||
epoch_pattern="^>"
|
||||
elif [ "$rinex_version" = "2" ]; then
|
||||
echo "Detected: RINEX 2.x format"
|
||||
epoch_pattern="^ [0-9][0-9] [ 0-9][0-9] [ 0-9][0-9]"
|
||||
else
|
||||
echo "Warning: Could not detect RINEX version, assuming 2.x"
|
||||
epoch_pattern="^ [0-9][0-9] [ 0-9][0-9] [ 0-9][0-9]"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
# Copy header from first file
|
||||
head -n $header_lines "$first_file" > "$output"
|
||||
echo "✓ Added header from $first_file"
|
||||
|
||||
# Append data from all files (skip their headers)
|
||||
total_lines=0
|
||||
for file in $files; do
|
||||
data_start=$(($(grep -n "END OF HEADER" "$file" | cut -d: -f1) + 1))
|
||||
data_lines=$(tail -n +$data_start "$file" | wc -l)
|
||||
tail -n +$data_start "$file" >> "$output"
|
||||
total_lines=$((total_lines + data_lines))
|
||||
echo "✓ Added data from $file ($data_lines lines)"
|
||||
done
|
||||
|
||||
echo ""
|
||||
|
||||
# Count epochs based on detected version
|
||||
if [ "$rinex_version" = "3" ]; then
|
||||
epochs=$(grep -c "^>" "$output" 2>/dev/null || echo "0")
|
||||
else
|
||||
epochs=$(grep -c "^ [0-9][0-9] [ 0-9][0-9] [ 0-9][0-9]" "$output" 2>/dev/null || echo "0")
|
||||
fi
|
||||
|
||||
# Get file size
|
||||
file_size=$(ls -lh "$output" | awk '{print $5}')
|
||||
|
||||
echo "========================================="
|
||||
echo "✓ Combined file created: $output"
|
||||
echo " RINEX Version: $rinex_version.x"
|
||||
echo " Size: $file_size"
|
||||
echo " Files combined: $file_count"
|
||||
echo " Total data lines: $total_lines"
|
||||
echo " Total epochs: $epochs"
|
||||
echo "========================================="
|
||||
|
||||
# Verify reasonable results
|
||||
if [ $epochs -eq 0 ]; then
|
||||
echo ""
|
||||
echo "⚠ WARNING: No epochs detected!"
|
||||
echo " This may indicate a problem with the combined file."
|
||||
echo " Checking first few lines of output..."
|
||||
echo ""
|
||||
head -40 "$output"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Show time range if epochs found
|
||||
if [ $epochs -gt 0 ]; then
|
||||
echo ""
|
||||
echo "Time range:"
|
||||
|
||||
if [ "$rinex_version" = "3" ]; then
|
||||
# RINEX 3.x format: > YYYY MM DD HH MM SS.SSSSSSS
|
||||
first_epoch=$(grep "^>" "$output" | head -1)
|
||||
last_epoch=$(grep "^>" "$output" | tail -1)
|
||||
first_time=$(echo "$first_epoch" | awk '{printf "%s-%02d-%02d %02d:%02d:%06.3f", $2, $3, $4, $5, $6, $7}')
|
||||
last_time=$(echo "$last_epoch" | awk '{printf "%s-%02d-%02d %02d:%02d:%06.3f", $2, $3, $4, $5, $6, $7}')
|
||||
else
|
||||
# RINEX 2.x format: YY MM DD HH MM SS.SSSSSSS
|
||||
first_epoch=$(grep "^ [0-9][0-9] [ 0-9][0-9] [ 0-9][0-9]" "$output" | head -1)
|
||||
last_epoch=$(grep "^ [0-9][0-9] [ 0-9][0-9] [ 0-9][0-9]" "$output" | tail -1)
|
||||
first_time=$(echo "$first_epoch" | awk '{printf "20%02d-%02d-%02d %02d:%02d:%011.7f", $1, $2, $3, $4, $5, $6}')
|
||||
last_time=$(echo "$last_epoch" | awk '{printf "20%02d-%02d-%02d %02d:%02d:%011.7f", $1, $2, $3, $4, $5, $6}')
|
||||
fi
|
||||
|
||||
echo " First: $first_time"
|
||||
echo " Last: $last_time"
|
||||
fi
|
||||
|
||||
# Verify single header (no corruption)
|
||||
header_count=$(grep -c "END OF HEADER" "$output")
|
||||
if [ $header_count -ne 1 ]; then
|
||||
echo ""
|
||||
echo "⚠ WARNING: Found $header_count headers (expected 1)"
|
||||
echo " The file may be corrupted!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "✓ File validation passed"
|
||||
1584
process_gps.py
Normal file
1584
process_gps.py
Normal file
File diff suppressed because it is too large
Load diff
3
run.sh
Executable file
3
run.sh
Executable file
|
|
@ -0,0 +1,3 @@
|
|||
source ./venv/bin/activate
|
||||
python3 process_gps.py -h
|
||||
python3 process_gps.py 2026-7-23_124722_serial-COM7.ubx
|
||||
193
setup.sh
Executable file
193
setup.sh
Executable file
|
|
@ -0,0 +1,193 @@
|
|||
#!/usr/bin/env bash
|
||||
# setup.sh - One-time environment bootstrap for the GPS processing pipeline.
|
||||
#
|
||||
# Installs (all local to this directory, no sudo required):
|
||||
# venv/ Python env: pyubx2, pygnssutils, pyserial, numpy, pandas, pyproj
|
||||
# tools/bin/ RTKLIB demo5 v2.5.1 (convbin, rnx2rtkp, pos2kml) [stock apt fallback]
|
||||
# tools/antex/ igs20.atx + ngs20.atx antenna models; tools/antenna.json (rover antenna)
|
||||
# tools/bin/CRX2RNX Hatanaka RINEX decompressor (fallback for .d CORS files)
|
||||
# tools/cache/ NGS CORS station catalog
|
||||
#
|
||||
# Idempotent: re-running skips anything already installed.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
TOOLS="$SCRIPT_DIR/tools"
|
||||
VENV="$SCRIPT_DIR/venv"
|
||||
DEMO5_TAG="v2.5.1"
|
||||
DEMO5_REPO="https://github.com/rtklibexplorer/RTKLIB"
|
||||
|
||||
PASS=()
|
||||
WARN=()
|
||||
|
||||
say() { printf '\n=== %s ===\n' "$*"; }
|
||||
ok() { printf ' [ok] %s\n' "$*"; PASS+=("$*"); }
|
||||
warn() { printf ' [WARN] %s\n' "$*"; WARN+=("$*"); }
|
||||
skip() { printf ' [skip] %s (already present)\n' "$*"; }
|
||||
|
||||
fetch() { # fetch <url> <dest>
|
||||
curl -fsSL --max-time 300 -o "$2" "$1"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- 1. Python venv
|
||||
say "1/6 Python virtual environment"
|
||||
if [ -x "$VENV/bin/python" ]; then
|
||||
skip "venv"
|
||||
else
|
||||
python3 -m venv "$VENV" || { echo "FATAL: venv creation failed"; exit 1; }
|
||||
fi
|
||||
if "$VENV/bin/python" -c "import pyubx2, pygnssutils, serial, numpy, pandas, pyproj, matplotlib" 2>/dev/null; then
|
||||
skip "python packages"
|
||||
else
|
||||
"$VENV/bin/pip" install --quiet --upgrade pip
|
||||
"$VENV/bin/pip" install --quiet pyubx2 pygnssutils pyserial numpy pandas pyproj matplotlib \
|
||||
|| { echo "FATAL: pip install failed"; exit 1; }
|
||||
fi
|
||||
"$VENV/bin/python" -c "import pyubx2, pygnssutils, serial, numpy, pandas, pyproj, matplotlib" \
|
||||
&& ok "venv with pyubx2/pygnssutils/pyserial/numpy/pandas/pyproj/matplotlib" \
|
||||
|| { echo "FATAL: package import check failed"; exit 1; }
|
||||
|
||||
mkdir -p "$TOOLS/bin" "$TOOLS/antex" "$TOOLS/cache" "$TOOLS/src"
|
||||
|
||||
# ---------------------------------------------------------------- 2. RTKLIB demo5
|
||||
say "2/6 RTKLIB demo5 $DEMO5_TAG (convbin, rnx2rtkp, pos2kml)"
|
||||
build_demo5_app() { # build_demo5_app <appname>
|
||||
local app="$1"
|
||||
local gccdir="$TOOLS/src/RTKLIB/app/consapp/$app/gcc"
|
||||
[ -d "$gccdir" ] || return 1
|
||||
# -lgfortran in some makefiles is vestigial; try without it first.
|
||||
if ! make -C "$gccdir" -j"$(nproc)" LDLIBS="-lm" >"$TOOLS/src/build_$app.log" 2>&1; then
|
||||
make -C "$gccdir" clean >/dev/null 2>&1
|
||||
make -C "$gccdir" -j"$(nproc)" >>"$TOOLS/src/build_$app.log" 2>&1 || return 1
|
||||
fi
|
||||
cp "$gccdir/$app" "$TOOLS/bin/$app"
|
||||
}
|
||||
|
||||
if [ -x "$TOOLS/bin/convbin" ] && [ -x "$TOOLS/bin/rnx2rtkp" ] && [ -x "$TOOLS/bin/pos2kml" ]; then
|
||||
skip "RTKLIB binaries"
|
||||
[ -f "$TOOLS/STOCK_RTKLIB" ] && warn "using STOCK RTKLIB (BDS/L5 signals will be missing from RINEX)"
|
||||
else
|
||||
if [ ! -d "$TOOLS/src/RTKLIB" ]; then
|
||||
git clone --depth 1 --branch "$DEMO5_TAG" "$DEMO5_REPO" "$TOOLS/src/RTKLIB" \
|
||||
|| warn "git clone of demo5 RTKLIB failed"
|
||||
fi
|
||||
built=true
|
||||
for app in convbin rnx2rtkp pos2kml; do
|
||||
if build_demo5_app "$app"; then
|
||||
printf ' built %s\n' "$app"
|
||||
else
|
||||
warn "demo5 build failed for $app (see tools/src/build_$app.log)"
|
||||
built=false
|
||||
fi
|
||||
done
|
||||
if $built; then
|
||||
rm -f "$TOOLS/STOCK_RTKLIB"
|
||||
ok "RTKLIB demo5 $DEMO5_TAG built from source"
|
||||
else
|
||||
# Fallback: stock RTKLIB 2.4.3 from the Ubuntu archive (works, but silently
|
||||
# drops BeiDou + L5/E5 observations from this receiver's raw data).
|
||||
echo " falling back to stock RTKLIB 2.4.3 from apt archive..."
|
||||
tmpd="$(mktemp -d)"
|
||||
if (cd "$tmpd" && apt-get download rtklib >/dev/null 2>&1 && dpkg -x rtklib_*.deb x); then
|
||||
cp "$tmpd"/x/usr/bin/{convbin,rnx2rtkp,pos2kml} "$TOOLS/bin/"
|
||||
touch "$TOOLS/STOCK_RTKLIB"
|
||||
warn "STOCK RTKLIB installed - BDS/L5 will be MISSING from RINEX output"
|
||||
else
|
||||
warn "stock RTKLIB fallback also failed - pipeline cannot run"
|
||||
fi
|
||||
rm -rf "$tmpd"
|
||||
fi
|
||||
fi
|
||||
for app in convbin rnx2rtkp pos2kml; do
|
||||
[ -x "$TOOLS/bin/$app" ] || warn "$app missing from tools/bin"
|
||||
done
|
||||
|
||||
# ---------------------------------------------------------------- 3. Antenna models
|
||||
say "3/6 Antenna calibration models (ANTEX)"
|
||||
if [ -s "$TOOLS/antex/igs20.atx" ]; then skip "igs20.atx"; else
|
||||
fetch "https://files.igs.org/pub/station/general/igs20.atx" "$TOOLS/antex/igs20.atx" \
|
||||
&& ok "igs20.atx (IGS antenna models incl. CORS base antennas)" \
|
||||
|| warn "igs20.atx download failed"
|
||||
fi
|
||||
if [ -s "$TOOLS/antex/ngs20.atx" ]; then skip "ngs20.atx"; else
|
||||
fetch "https://geodesy.noaa.gov/ANTCAL/LoadFile?file=ngs20.atx" "$TOOLS/antex/ngs20.atx" \
|
||||
&& ok "ngs20.atx (NGS absolute calibrations incl. ArduSimple)" \
|
||||
|| warn "ngs20.atx download failed"
|
||||
fi
|
||||
# Resolve the rover antenna (NGS calibration AS-ANT3BCAL01, ArduSimple) to its
|
||||
# exact 20-char ANTEX type string, consumed by process_gps.py as ant1-anttype.
|
||||
"$VENV/bin/python" - "$TOOLS" <<'EOF'
|
||||
import json, re, sys, pathlib
|
||||
tools = pathlib.Path(sys.argv[1])
|
||||
result = {"antex_name": None, "source": None, "candidates": []}
|
||||
for atx in ("ngs20.atx", "igs20.atx"):
|
||||
p = tools / "antex" / atx
|
||||
if not p.is_file():
|
||||
continue
|
||||
for line in p.open(encoding="utf-8", errors="replace"):
|
||||
if "TYPE / SERIAL" in line and "ANT3B" in line[:20].upper():
|
||||
name = line[:20].rstrip()
|
||||
result["candidates"].append({"name": name, "file": atx})
|
||||
if result["antex_name"] is None:
|
||||
result["antex_name"], result["source"] = name, atx
|
||||
(tools / "antenna.json").write_text(json.dumps(result, indent=2))
|
||||
if result["antex_name"]:
|
||||
print(f' [ok] rover antenna resolved: "{result["antex_name"]}" ({result["source"]})')
|
||||
else:
|
||||
print(" [WARN] no ANT3B antenna found in ANTEX files - PPK will run without rover PCO/PCV")
|
||||
EOF
|
||||
|
||||
# ---------------------------------------------------------------- 4. CRX2RNX
|
||||
say "4/6 CRX2RNX (Hatanaka RINEX decompressor, fallback for .d CORS files)"
|
||||
if [ -x "$TOOLS/bin/CRX2RNX" ]; then
|
||||
skip "CRX2RNX"
|
||||
else
|
||||
tmpd="$(mktemp -d)"
|
||||
if fetch "https://terras.gsi.go.jp/ja/crx2rnx/RNXCMP_4.2.0_Linux_gcc_64bit.tar.gz" "$tmpd/rnxcmp.tar.gz" \
|
||||
&& tar -xzf "$tmpd/rnxcmp.tar.gz" -C "$tmpd" \
|
||||
&& cp "$tmpd"/RNXCMP_*/bin/CRX2RNX "$TOOLS/bin/CRX2RNX" 2>/dev/null \
|
||||
|| cp "$tmpd"/RNXCMP_*/CRX2RNX "$TOOLS/bin/CRX2RNX" 2>/dev/null; then
|
||||
chmod +x "$TOOLS/bin/CRX2RNX"
|
||||
ok "CRX2RNX 4.2.0"
|
||||
else
|
||||
warn "CRX2RNX install failed (only needed for CORS stations without plain .o files)"
|
||||
fi
|
||||
rm -rf "$tmpd"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------- 5. CORS catalog
|
||||
say "5/6 NGS CORS station catalog"
|
||||
if [ -s "$TOOLS/cache/itrf2020_geo.comp.txt" ]; then
|
||||
skip "station catalog"
|
||||
else
|
||||
fetch "https://geodesy.noaa.gov/corsdata/coord/coord_20/itrf2020_geo.comp.txt" \
|
||||
"$TOOLS/cache/itrf2020_geo.comp.txt" \
|
||||
&& ok "CORS station catalog (ITRF2020 coordinates)" \
|
||||
|| warn "CORS catalog download failed (process_gps.py will retry at run time)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------- 6. Self-check
|
||||
say "6/6 Self-check summary"
|
||||
for app in convbin rnx2rtkp pos2kml; do
|
||||
if [ -x "$TOOLS/bin/$app" ]; then
|
||||
printf ' %-10s %s\n' "$app" "$("$TOOLS/bin/$app" -h 2>&1 | grep -m1 -iE 'synops|usage|version' || echo present)"
|
||||
fi
|
||||
done
|
||||
[ -x "$TOOLS/bin/CRX2RNX" ] && printf ' %-10s %s\n' "CRX2RNX" "present"
|
||||
[ -s "$TOOLS/cache/itrf2020_geo.comp.txt" ] && printf ' %-10s %d stations\n' "catalog" "$(wc -l < "$TOOLS/cache/itrf2020_geo.comp.txt")"
|
||||
[ -s "$TOOLS/antenna.json" ] && printf ' %-10s %s\n' "antenna" "$(cat "$TOOLS/antenna.json" | "$VENV/bin/python" -c 'import json,sys; print(json.load(sys.stdin)["antex_name"])')"
|
||||
|
||||
echo
|
||||
if [ ${#WARN[@]} -gt 0 ]; then
|
||||
echo "Completed with ${#WARN[@]} warning(s):"
|
||||
printf ' - %s\n' "${WARN[@]}"
|
||||
else
|
||||
echo "Setup complete - no warnings."
|
||||
fi
|
||||
echo
|
||||
echo "Mode 1 (real-time RTK) additionally requires free registration at"
|
||||
echo " http://rtn.usm.edu/RegisterAccount.aspx (Mississippi GCGC RTN)"
|
||||
echo "then: export GCGC_USER=<user> GCGC_PASS=<pass>"
|
||||
446
stream_gps.py
Normal file
446
stream_gps.py
Normal file
|
|
@ -0,0 +1,446 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
stream_gps.py - Mode 1: continuous real-time positioning for the u-blox ZED-X20P.
|
||||
|
||||
Best-available correction tier is selected automatically:
|
||||
1. Network RTK via NTRIP (default: Mississippi GCGC RTN, free registration at
|
||||
http://rtn.usm.edu/RegisterAccount.aspx) -> onboard RTK engine, 1-3 cm.
|
||||
Credentials: env GCGC_USER / GCGC_PASS (or --ntrip-user/--ntrip-pass).
|
||||
2. Galileo HAS (E6-B, no internet needed) -> ~20-25 cm. Requires firmware
|
||||
HPG >= 2.10 (auto-probed via MON-VER; this project's module reports 2.10).
|
||||
3. Neither available -> the onboard multiband+SBAS solution (~1 m) cannot be
|
||||
improved in real time; per project policy the enhanced mode is SKIPPED
|
||||
(run with --log-only to still record a .ubx for post-processing).
|
||||
|
||||
Every session simultaneously logs raw RXM-RAWX/SFRBX to a .ubx file, so a
|
||||
streaming session can always be post-processed later with process_gps.py.
|
||||
|
||||
Buoy deployment defaults: the receiver's dynamic platform model is set to "sea"
|
||||
(--dynmodel to override) and the NTRIP GGA uplink sends the LIVE position so the
|
||||
VRS reference follows platform drift (--gga-fixed to disable for a fixed antenna).
|
||||
|
||||
Outputs: live console status, stream_<ts>.csv (decimal-degree lat/lon),
|
||||
stream_<ts>.ubx (raw capture).
|
||||
|
||||
Usage:
|
||||
venv/bin/python stream_gps.py --port /dev/ttyUSB0 (Linux)
|
||||
venv/bin/python stream_gps.py --port COM7 (Windows)
|
||||
venv/bin/python stream_gps.py --replay capture.ubx (offline test)
|
||||
|
||||
Datum note: NTRIP corrections define the output frame. GCGC RTN broadcasts
|
||||
NAD83(2011) epoch 2010.00 - constant ~1.5 m from WGS84/ITRF in CONUS; the CSV
|
||||
header records which tier (and therefore frame) was active.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import os
|
||||
import queue
|
||||
import signal
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from collections import Counter
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from pyubx2 import UBXMessage, UBXReader, UBX_PROTOCOL, SET_LAYER_RAM
|
||||
|
||||
FIX_NAME = {0: "none", 1: "DR", 2: "2D", 3: "3D", 4: "GNSS+DR", 5: "time-only"}
|
||||
CARR_NAME = {0: "", 1: "RTK-FLOAT", 2: "RTK-FIXED"}
|
||||
|
||||
# UBX-CFG keys (u-blox X20 HPG interface descriptions; NAVCOR from HPG 2.10 JSON)
|
||||
K = {
|
||||
"UART1_BAUD": 0x40520001,
|
||||
"UART1_IN_RTCM3": 0x10730004,
|
||||
"USB_IN_RTCM3": 0x10770004,
|
||||
"PVT_UART1": 0x20910007, "PVT_USB": 0x20910009,
|
||||
"HPPOSLLH_UART1": 0x20910034, "HPPOSLLH_USB": 0x20910036,
|
||||
"RAWX_UART1": 0x209102a5, "RAWX_USB": 0x209102a7,
|
||||
"SFRBX_UART1": 0x20910232, "SFRBX_USB": 0x20910234,
|
||||
"NAVCOR_ENABLE_HOST": 0x100d0001, # HPG >= 2.10
|
||||
"NAVCOR_ENABLE_GAL_HAS": 0x100d0002, # HPG >= 2.10
|
||||
"DYNMODEL": 0x20110021, # CFG-NAVSPG-DYNMODEL
|
||||
}
|
||||
|
||||
# CFG-NAVSPG-DYNMODEL values; default "sea" for the buoy deployment
|
||||
DYNMODEL = {"portable": 0, "stationary": 2, "pedestrian": 3, "automotive": 4,
|
||||
"sea": 5, "airborne1g": 6, "airborne2g": 7, "airborne4g": 8, "wrist": 9}
|
||||
|
||||
|
||||
def log(msg):
|
||||
print(f"[stream_gps] {msg}", flush=True)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Receiver configuration
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
def valset(ser, ubr, cfg, label):
|
||||
"""Send CFG-VALSET (RAM) and wait briefly for ACK; returns True on ACK-ACK."""
|
||||
msg = UBXMessage.config_set(layers=SET_LAYER_RAM, transaction=0, cfgData=cfg)
|
||||
ser.write(msg.serialize())
|
||||
t_end = time.time() + 2.0
|
||||
while time.time() < t_end:
|
||||
try:
|
||||
_, parsed = ubr.read()
|
||||
except Exception:
|
||||
continue
|
||||
if parsed is None:
|
||||
continue
|
||||
if parsed.identity == "ACK-ACK":
|
||||
return True
|
||||
if parsed.identity == "ACK-NAK":
|
||||
log(f" NAK for {label}")
|
||||
return False
|
||||
log(f" no ACK for {label} (continuing)")
|
||||
return False
|
||||
|
||||
|
||||
def poll_fw_version(ser, ubr):
|
||||
"""Poll MON-VER; returns firmware string like 'HPG 2.10' or None."""
|
||||
ser.write(UBXMessage("MON", "MON-VER", 2).serialize()) # 2 = POLL msgmode
|
||||
t_end = time.time() + 3.0
|
||||
while time.time() < t_end:
|
||||
try:
|
||||
_, parsed = ubr.read()
|
||||
except Exception:
|
||||
continue
|
||||
if parsed is not None and parsed.identity == "MON-VER":
|
||||
sw = getattr(parsed, "swVersion", b"")
|
||||
sw = sw.decode(errors="replace") if isinstance(sw, bytes) else str(sw)
|
||||
return sw.strip("\x00 ")
|
||||
return None
|
||||
|
||||
|
||||
def configure_receiver(ser, ubr, want_has, dynmodel):
|
||||
"""Enable RTCM input, HP output, raw logging; optionally HAS. RAM layer only."""
|
||||
base_cfg = [
|
||||
(K["UART1_IN_RTCM3"], 1), (K["USB_IN_RTCM3"], 1),
|
||||
(K["PVT_UART1"], 1), (K["PVT_USB"], 1),
|
||||
(K["HPPOSLLH_UART1"], 1), (K["HPPOSLLH_USB"], 1),
|
||||
(K["RAWX_UART1"], 1), (K["RAWX_USB"], 1),
|
||||
(K["SFRBX_UART1"], 1), (K["SFRBX_USB"], 1),
|
||||
(K["DYNMODEL"], DYNMODEL[dynmodel]),
|
||||
]
|
||||
ok = valset(ser, ubr, base_cfg, f"base output/input config (dynmodel={dynmodel})")
|
||||
if want_has:
|
||||
ok_has = valset(ser, ubr, [(K["NAVCOR_ENABLE_HOST"], 0),
|
||||
(K["NAVCOR_ENABLE_GAL_HAS"], 1)], "GAL HAS enable")
|
||||
return ok, ok_has
|
||||
return ok, False
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# NTRIP
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
def caster_reachable(server, port, timeout=8):
|
||||
try:
|
||||
with socket.create_connection((server, port), timeout=timeout):
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
class SerialRTCMSink:
|
||||
"""Write-only wrapper handed to GNSSNTRIPClient as its output stream."""
|
||||
def __init__(self, ser):
|
||||
self._ser = ser
|
||||
self.bytes_out = 0
|
||||
|
||||
def write(self, data):
|
||||
# GNSSNTRIPClient may hand us (raw, parsed) tuples or raw bytes
|
||||
if isinstance(data, tuple):
|
||||
data = data[0]
|
||||
if isinstance(data, (bytes, bytearray)):
|
||||
self._ser.write(bytes(data))
|
||||
self.bytes_out += len(data)
|
||||
|
||||
# queue-style interface (pygnssutils uses .put on Queue outputs)
|
||||
def put(self, item):
|
||||
self.write(item)
|
||||
|
||||
|
||||
def start_ntrip(args, sink, reflat, reflon, app=None):
|
||||
"""app (with .get_coordinates()) => live GGA that follows a drifting platform;
|
||||
without app, falls back to a fixed reference position."""
|
||||
from pygnssutils import GNSSNTRIPClient
|
||||
client = GNSSNTRIPClient(app=app)
|
||||
kwargs = dict(
|
||||
server=args.ntrip_caster, port=args.ntrip_port, https=0,
|
||||
mountpoint=args.ntrip_mount, datatype="RTCM",
|
||||
ntripuser=args.ntrip_user, ntrippassword=args.ntrip_pass,
|
||||
ggainterval=args.gga_interval,
|
||||
ggamode=0 if app is not None else 1, # 0 = live from receiver, 1 = fixed
|
||||
reflat=reflat, reflon=reflon, refalt=0.0, refsep=0.0,
|
||||
output=sink,
|
||||
)
|
||||
ok = client.run(**kwargs)
|
||||
return client if ok else None
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Main streaming loop
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
class Session:
|
||||
def __init__(self, args):
|
||||
self.args = args
|
||||
self.q = queue.Queue()
|
||||
self.stop = threading.Event()
|
||||
self.fix_hist = Counter()
|
||||
self.hacc = []
|
||||
self.tier = "none"
|
||||
# latest fix, served to GNSSNTRIPClient.get_coordinates() for live GGA
|
||||
self.live = {"lat": 0.0, "lon": 0.0, "alt": 0.0, "sep": 0.0}
|
||||
ts = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
|
||||
self.csv_path = Path(f"stream_{ts}.csv")
|
||||
self.ubx_path = Path(f"stream_{ts}.ubx")
|
||||
|
||||
def get_coordinates(self):
|
||||
"""pygnssutils app interface: live position for the NTRIP GGA uplink,
|
||||
so the VRS reference follows a drifting platform (buoy)."""
|
||||
return dict(self.live)
|
||||
|
||||
def reader_thread(self, stream, raw_log):
|
||||
"""Serial/file -> raw .ubx tee -> parsed NAV messages -> queue."""
|
||||
ubr = UBXReader(stream, protfilter=UBX_PROTOCOL, quitonerror=0)
|
||||
while not self.stop.is_set():
|
||||
try:
|
||||
raw, parsed = ubr.read()
|
||||
except Exception:
|
||||
if self.args.replay:
|
||||
break
|
||||
continue
|
||||
if raw is None:
|
||||
if self.args.replay:
|
||||
break
|
||||
continue
|
||||
if raw_log:
|
||||
raw_log.write(raw)
|
||||
if parsed is None:
|
||||
continue
|
||||
if parsed.identity in ("NAV-PVT", "NAV-HPPOSLLH"):
|
||||
self.q.put(parsed)
|
||||
self.q.put(None)
|
||||
|
||||
def run(self):
|
||||
args = self.args
|
||||
# ---------------- input: serial or replay file ----------------------
|
||||
if args.replay:
|
||||
stream = open(args.replay, "rb")
|
||||
ser = None
|
||||
log(f"REPLAY mode from {args.replay} (no config writes, no NTRIP)")
|
||||
else:
|
||||
import serial
|
||||
from serial.tools import list_ports
|
||||
if not args.port:
|
||||
ports = [p.device for p in list_ports.comports()]
|
||||
sys.exit(f"--port required. Detected ports: {ports or 'none'}")
|
||||
ser = serial.Serial(args.port, args.baud, timeout=1)
|
||||
stream = ser
|
||||
log(f"connected {args.port} @ {args.baud}")
|
||||
|
||||
cfg_ubr = UBXReader(ser, protfilter=UBX_PROTOCOL, quitonerror=0)
|
||||
fw = poll_fw_version(ser, cfg_ubr)
|
||||
log(f"firmware: {fw or 'unknown'}")
|
||||
fw_has = bool(fw and any(
|
||||
fw.split("HPG")[-1].strip().startswith(v)
|
||||
for v in ("2.1", "2.2", "3.")))
|
||||
|
||||
# ------------- correction tier selection -------------------------
|
||||
ntrip_ok = False
|
||||
if not args.log_only and args.ntrip_user and args.ntrip_pass:
|
||||
ntrip_ok = caster_reachable(args.ntrip_caster, args.ntrip_port)
|
||||
if not ntrip_ok:
|
||||
log(f"NTRIP caster {args.ntrip_caster}:{args.ntrip_port} unreachable")
|
||||
elif not args.log_only:
|
||||
log("no NTRIP credentials (set GCGC_USER/GCGC_PASS or --ntrip-user/-pass)")
|
||||
|
||||
want_has = not ntrip_ok and not args.log_only and fw_has
|
||||
if ntrip_ok:
|
||||
self.tier = f"NTRIP RTK ({args.ntrip_caster}/{args.ntrip_mount})"
|
||||
elif want_has:
|
||||
self.tier = "Galileo HAS (E6-B, ~20 cm)"
|
||||
elif args.log_only:
|
||||
self.tier = "log-only (onboard solution)"
|
||||
else:
|
||||
log("no correction source available that improves on the onboard "
|
||||
"solution (NTRIP unreachable/no credentials; HAS needs FW >= "
|
||||
"HPG 2.10). Skipping enhanced streaming per project policy.")
|
||||
log("use --log-only to record raw data for post-processing anyway.")
|
||||
ser.close()
|
||||
return 2
|
||||
|
||||
ok_base, ok_has = configure_receiver(ser, cfg_ubr, want_has, args.dynmodel)
|
||||
log(f"receiver configured (base={'ok' if ok_base else 'partial'}"
|
||||
+ (f", HAS={'ok' if ok_has else 'failed'}" if want_has else "") + ")")
|
||||
|
||||
log(f"correction tier: {self.tier}")
|
||||
|
||||
# ---------------- outputs -------------------------------------------
|
||||
raw_log = None if args.replay else open(self.ubx_path, "wb")
|
||||
csv_f = open(self.csv_path, "w", newline="")
|
||||
csv_f.write(f"# stream_gps.py session - correction tier: {self.tier}\n")
|
||||
csv_f.write("# datum: NAD83(2011) if GCGC NTRIP tier, else WGS84/ITRF\n")
|
||||
writer = csv.writer(csv_f)
|
||||
writer.writerow(["utc", "lat_deg", "lon_deg", "ellip_height_m",
|
||||
"hAcc_m", "vAcc_m", "fixType", "carrSoln", "numSV"])
|
||||
|
||||
# ---------------- threads -------------------------------------------
|
||||
t_read = threading.Thread(target=self.reader_thread, args=(stream, raw_log),
|
||||
daemon=True)
|
||||
t_read.start()
|
||||
|
||||
ntrip_client, sink = None, None
|
||||
if ser is not None and self.tier.startswith("NTRIP"):
|
||||
# need a first fix for the GGA reference position
|
||||
reflat, reflon = args.lat, args.lon
|
||||
if reflat is None:
|
||||
log("waiting for first fix to seed NTRIP GGA position ...")
|
||||
first = self._wait_first_fix(timeout=60)
|
||||
if first is None:
|
||||
log("no fix within 60 s; using 0/0 GGA (VRS may reject)")
|
||||
reflat = reflon = 0.0
|
||||
else:
|
||||
reflat, reflon = first
|
||||
sink = SerialRTCMSink(ser)
|
||||
app = None if args.gga_fixed else self
|
||||
log("GGA uplink: " + ("fixed reference" if app is None
|
||||
else "live (follows platform drift)"))
|
||||
ntrip_client = start_ntrip(args, sink, reflat, reflon, app=app)
|
||||
if ntrip_client is None:
|
||||
log("NTRIP connection failed (check credentials/mountpoint) - "
|
||||
"continuing with onboard solution only")
|
||||
self.tier += " [FAILED - onboard only]"
|
||||
|
||||
signal.signal(signal.SIGINT, lambda *_: self.stop.set())
|
||||
|
||||
# ---------------- consume + display ---------------------------------
|
||||
last_pvt = {}
|
||||
try:
|
||||
while not self.stop.is_set():
|
||||
try:
|
||||
m = self.q.get(timeout=1.0)
|
||||
except queue.Empty:
|
||||
continue
|
||||
if m is None:
|
||||
break
|
||||
if m.identity == "NAV-PVT":
|
||||
last_pvt = dict(
|
||||
utc=datetime(m.year, m.month, m.day, m.hour, m.min, m.second,
|
||||
tzinfo=timezone.utc),
|
||||
fixType=m.fixType, carrSoln=m.carrSoln, numSV=m.numSV,
|
||||
lat=m.lat, lon=m.lon, height=m.height / 1000.0,
|
||||
hAcc=m.hAcc / 1000.0, vAcc=m.vAcc / 1000.0)
|
||||
elif m.identity == "NAV-HPPOSLLH" and last_pvt:
|
||||
# high-precision refinement: lat/lon + Hp components (deg)
|
||||
last_pvt["lat"] = m.lat + m.latHp
|
||||
last_pvt["lon"] = m.lon + m.lonHp
|
||||
last_pvt["height"] = (m.height + m.heightHp) / 1000.0
|
||||
last_pvt["hAcc"] = m.hAcc / 1000.0
|
||||
last_pvt["vAcc"] = m.vAcc / 1000.0
|
||||
continue
|
||||
if not last_pvt:
|
||||
continue
|
||||
p = last_pvt
|
||||
self.live.update(lat=p["lat"], lon=p["lon"], alt=p["height"])
|
||||
self.fix_hist[(p["fixType"], p["carrSoln"])] += 1
|
||||
self.hacc.append(p["hAcc"])
|
||||
writer.writerow([p["utc"].isoformat(), f"{p['lat']:.9f}",
|
||||
f"{p['lon']:.9f}", f"{p['height']:.3f}",
|
||||
f"{p['hAcc']:.3f}", f"{p['vAcc']:.3f}",
|
||||
p["fixType"], p["carrSoln"], p["numSV"]])
|
||||
n_epochs = sum(self.fix_hist.values())
|
||||
if not sys.stdout.isatty() and n_epochs % 60 != 1:
|
||||
continue # avoid log spam when piped
|
||||
fixlab = FIX_NAME.get(p["fixType"], "?")
|
||||
if p["carrSoln"]:
|
||||
fixlab += "/" + CARR_NAME[p["carrSoln"]]
|
||||
rtcm = f" rtcm {sink.bytes_out//1024} kB" if sink else ""
|
||||
end = " " if sys.stdout.isatty() else "\n"
|
||||
status = (f"\r{p['utc']:%H:%M:%S} {fixlab:<12s} "
|
||||
f"{p['lat']:+.9f} {p['lon']:+.9f} h={p['height']:8.3f} m "
|
||||
f"hAcc={p['hAcc']:6.3f} m sv={p['numSV']:2d}{rtcm}{end}")
|
||||
sys.stdout.write(status)
|
||||
sys.stdout.flush()
|
||||
finally:
|
||||
self.stop.set()
|
||||
print()
|
||||
if ntrip_client is not None:
|
||||
try:
|
||||
ntrip_client.stop()
|
||||
except Exception:
|
||||
pass
|
||||
csv_f.close()
|
||||
if raw_log:
|
||||
raw_log.close()
|
||||
if ser:
|
||||
ser.close()
|
||||
self._summary()
|
||||
return 0
|
||||
|
||||
def _wait_first_fix(self, timeout):
|
||||
t_end = time.time() + timeout
|
||||
while time.time() < t_end:
|
||||
try:
|
||||
m = self.q.get(timeout=1.0)
|
||||
except queue.Empty:
|
||||
continue
|
||||
if m is not None and m.identity == "NAV-PVT" and m.fixType >= 2:
|
||||
self.live.update(lat=m.lat, lon=m.lon, alt=m.height / 1000.0)
|
||||
return (m.lat, m.lon)
|
||||
return None
|
||||
|
||||
def _summary(self):
|
||||
n = sum(self.fix_hist.values())
|
||||
if not n:
|
||||
log("no position epochs received")
|
||||
return
|
||||
log(f"session summary: {n} epochs, tier = {self.tier}")
|
||||
for (fix, carr), c in sorted(self.fix_hist.items(), key=lambda kv: -kv[1]):
|
||||
lab = FIX_NAME.get(fix, "?") + ("/" + CARR_NAME[carr] if carr else "")
|
||||
log(f" {lab:<14s} {c:6d} ({100*c/n:.1f}%)")
|
||||
h = sorted(self.hacc)
|
||||
log(f" hAcc: mean {sum(h)/len(h):.3f} m, median {h[len(h)//2]:.3f} m, "
|
||||
f"95% {h[int(len(h)*0.95)]:.3f} m")
|
||||
if not self.args.replay:
|
||||
log(f" outputs: {self.csv_path}, {self.ubx_path}")
|
||||
log(f" post-process the raw log: venv/bin/python process_gps.py {self.ubx_path}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--port", help="serial port (COM7, /dev/ttyUSB0, ...)")
|
||||
ap.add_argument("--baud", type=int, default=38400)
|
||||
ap.add_argument("--replay", type=Path, help="read a .ubx file instead of serial "
|
||||
"(offline parser/CSV test mode)")
|
||||
ap.add_argument("--log-only", action="store_true",
|
||||
help="skip corrections; just log raw data + onboard positions")
|
||||
ap.add_argument("--lat", type=float, help="initial latitude for NTRIP GGA (else first fix)")
|
||||
ap.add_argument("--lon", type=float, help="initial longitude for NTRIP GGA")
|
||||
ap.add_argument("--gga-fixed", action="store_true",
|
||||
help="send a FIXED GGA reference instead of live position "
|
||||
"(only for a truly stationary antenna; default is live, "
|
||||
"so the VRS reference follows a drifting buoy)")
|
||||
ap.add_argument("--dynmodel", default="sea", choices=sorted(DYNMODEL),
|
||||
help="receiver dynamic platform model, set in RAM at startup "
|
||||
"(default: sea, for buoy deployment)")
|
||||
ap.add_argument("--ntrip-caster", default="rtn.usm.edu")
|
||||
ap.add_argument("--ntrip-port", type=int, default=2101)
|
||||
ap.add_argument("--ntrip-mount", default="VRS_GNSS_RTCM34_NAD83")
|
||||
ap.add_argument("--ntrip-user", default=os.environ.get("GCGC_USER"))
|
||||
ap.add_argument("--ntrip-pass", default=os.environ.get("GCGC_PASS"))
|
||||
ap.add_argument("--gga-interval", type=int, default=15)
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.replay and not args.port:
|
||||
ap.error("one of --port or --replay is required")
|
||||
sys.exit(Session(args).run())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Add table
Reference in a new issue