ergodic-sampling/findings.md

# Findings: sampling patterns, reconstruction methods, and what the ground truth is made of

Written 2026-09-12. Refer to [ergodic_sampling_test.py](ergodic_sampling_test.py) for analysis and figures.

## What was tested

The analysis uses a 64 by 64 grid and keeps only 11.8% of the cells (484 samples), placed either on a
sparse grid (every third cell) or in the "ergodic" irregular pattern from the Zhang and Li paper
(found by minimizing equation 4). From those samples we rebuild the full grid with several methods
and measure how much the reconstruction differs from the ground truth. The value of merit chosen is
root-mean-squared error: the typical size of the difference between the reconstruction and the true
field at a cell, normalized such that 1.0 is 1 standard deviation (lower is better). Each RMS error 
value shown is the average over 25 iterations with a freshly generated ground truth.

The reconstruction methods:

- **Spline interpolation**: fits a smooth surface through the sample points. This is the
  conventional baseline the paper also compares against.
- **Compressive sensing**: the paper's method, iterative thresholding that assumes the signal is
  built from a small number of components in some transform. It is run two ways: with a Fourier
  transform (components are sinusoidal waves) and with the symlet wavelet from the paper.
- **Kriging**: an interpolation method from geostatistics that the paper does not use; it was added
  as a strong reference. It estimates each empty cell as a weighted average of the samples, where
  the weights come from a model of how similar the field tends to be at each separation distance.
  That similarity model is itself estimated from the samples, so kriging adapts to the data: for a
  smooth field it interpolates broadly, and for structureless data it learns that the samples say
  nothing about their neighbors and backs off.

The three kinds of ground truth:

- **White noise (control)**: every cell is an independent random number. There is no structure at
  all, so nothing between the samples can actually be predicted. This serves as a good control
  because it emphasizes any tendency of the reconstruction to hallucinate structure based on its
  assumptions about the data.
- **Fourier-sparse**: a sum of 12 sinusoidal waves, half of them varying too quickly for the
  sparse grid to follow. This is exactly the kind of signal compressive sensing is designed for,
  and matches the paper's own demonstration signal.
- **Potential-field**: a smooth random map that mimics the paper's gravity survey data.

## The numbers

Root-mean-square error, mean plus or minus spread over 25 trials (best per column in bold):

| method | white noise (control) | Fourier-sparse | potential-field |
|---|---|---|---|
| sparse + spline | 1.221 ± 0.023 | 0.517 ± 0.066 | 0.069 ± 0.012 |
| sparse + compressive sensing (Fourier) | 1.076 ± 0.011 | 0.155 ± 0.155 | 0.866 ± 0.100 |
| sparse + kriging | 1.013 ± 0.009 | 0.485 ± 0.065 | **0.060 ± 0.010** |
| ergodic + spline | 1.445 ± 0.049 | 0.674 ± 0.078 | 0.166 ± 0.039 |
| ergodic + compressive sensing (Fourier) | 1.056 ± 0.010 | **0.000 ± 0.000** | 0.252 ± 0.042 |
| ergodic + compressive sensing (symlet) | 1.304 ± 0.034 | 0.737 ± 0.107 | 0.145 ± 0.020 |
| ergodic + kriging | **1.002 ± 0.007** | 0.558 ± 0.074 | 0.094 ± 0.016 |

## What was found

**1. On truly random ground truth, ergodic + kriging came out best of everything tested.**
When the field has no structure, the smartest possible move is to admit it: output zero at every
unsampled cell, which scores 0.94 on our scale. Ergodic + kriging lands essentially on that mark
(1.002) because its similarity model, estimated from the samples, correctly concludes the samples
carry no information about their neighbors, so it barely interpolates at all. Every method with a
built-in belief about structure does worse than the "give up" answer: spline draws smooth hills
that do not exist (1.22 on the sparse grid, 1.45 on the ergodic pattern) and the symlet version
paints in wavelet texture (1.30). That gap, up to roughly 50% worse than guessing zero, is the
cost of hallucinated structure made concrete.

**2. On the potential-field ground truth, the plain sparse grid is the best pattern.** This
field is smooth enough that every third cell is dense enough sampling, the situation classical
sampling theory covers. There the sparse grid wins simply on coverage: its farthest cell from
any sample is 1.41 cells away, while the ergodic pattern, being irregular, leaves gaps up to 3
cells. The ergodic pattern costs about 1.5 times the error here. That cost is the
premium for insurance that pays out on the Fourier-sparse truth, where half the signal varies too
fast for the sparse grid: there the sparse grid garbles the fast waves into false slow ones
(this is aliasing, and no processing can undo it). THIS CASE IS MOSTLY THEORETICAL: If we knew
that there were no small-scale variation to capture, the sparse grid would be the right choice
outright; in practice that is rarely known before sampling, which is exactly what motivates the
signal-agnostic ergodic pattern.

**3. Matching the reconstruction method to how the ground truth was generated wins, as expected.**
Each column of the table is won by the method whose built-in assumption mirrors the generator: 
kriging on the smooth random fields (its fitted similarity model actually recovers the
generator's parameters exactly), Fourier compressive sensing on the sum-of-waves signal, and
kriging again on noise because it alone can gracefully back off to predicting zero. Mismatches
fail hard: Fourier compressive sensing assumes a few dominant waves and scores 0.866 on the
smooth field, and the symlet version, which assumes the wrong kind of building block for a
sum-of-waves signal, reaches only 0.737 where the Fourier version is exact.

## The bigger picture

Choosing how to reconstruct in essence is a choice about how much you trust what you know about
the underlying signal. A correct guess about signal structure rewards all the way up to perfect
recovery with minimal samples when the assumption is exactly right. The same choices punish wrong
assumptions: a method that expects structure will manufacture it out of nothing, and on our 
structureless control every such method lost to simply predicting zero, with kriging only slightly
worse than predicting zero. The paper's ergodic pattern is best understood in this light: it is 
deliberately designed without any knowledge of the signal, as insurance that keeps every option open.
Alongside it, our error-prediction experiment (the plt2 figures) shows that a map of the expected 
error at every cell can be computed from nothing but the sample values and their locations, and 
it tracks the actual error pattern closely (correlation 0.91 to 0.99 against the error measured over 64
independent trials, with correctly calibrated magnitudes).

## Potential future work

Since we have shown that for cases where no small scale variation exists to capture, sparse sampling
wins due to optimal coverage; we know that in such a case Ergodic sampling would still be superior 
if we knew at what distance scale local variation existed and further reduced our sample count.

I propose the following:

Develop a sampling procedure which optimizes for collection cost in two collection scenarios
 - Where travel is the dominant cost
   - This applies to survey type collection where a vehicle carrying a sensor is used
   - Compute an optimal survey path which leverages Ergodic sensing methods but reframed as a
     densely sampled path; choosing the path dynamically as sensed signal spatial scales are discovered.

 - Where number of sample locations are the dominant cost
   - This applies to satellite-pointing type collection or ground-station collection
   - Compute an optimal survey path which leverages Ergodic sensing methods, but is scale-adaptive.
     Rather than having coverage sparsity as a prior, this would discover the required coverage
     dynamically to attain a certain reconstruction confidence metric.

Both scenarios should ideally attempt to iteratively estimate a confidence level based on how much future
predictions match the current model (created from previous samples), and therefore determine when sampling
should stop due to being sufficient for our confidence interval goals.