Getting started: end-to-end inference (Ornstein–Uhlenbeck)

synthetic overdamped linear 1D

The recommended starting point: the complete Stochastic Force Inference (SFI) workflow on the simplest non-trivial system, a particle in a harmonic trap. Its position obeys the overdamped Langevin (Ornstein–Uhlenbeck) equation

\[\mathrm{d}x = \underbrace{-k\,x}_{F(x)}\,\mathrm{d}t + \sqrt{2D}\;\mathrm{d}W ,\]

a restoring force \(F(x) = -k\,x\) plus white noise of strength \(D\). We simulate a trajectory, then pretend not to know \(F\) or \(D\) and recover them from the data alone:

  1. estimate \(D\) and fit \(F\) on a deliberately over-complete set of candidate terms, with error bars;

  2. let SFI select the minimal model;

  3. validate the fit as one would on real data — residual tests and a bootstrapped trajectory;

  4. save the model and load it back.

Real recordings follow the same path: load your trajectories instead of simulating them (Experimental-data workflow template).

Tags

synthetic · overdamped · linear · 1D

stamp_output()
[Generated: 2026-10-17 23:14]

The data

We build the exact model with OverdampedProcess — the force -k * X(dim=1) is \(-k\,x\), with X() the coordinate — and record 10 000 frames, i.e. 200 relaxation times \(1/k\).

import SFI
from SFI.bases import X
from SFI.langevin import OverdampedProcess

k_true, D_true = 1.0, 0.5   # trap stiffness and diffusion constant
dt = 0.02                   # time between recorded frames

proc = OverdampedProcess(F=-k_true * X(dim=1), D=D_true)
proc.initialize(jnp.array([0.0]))
coll = proc.simulate(dt=dt, Nsteps=10_000, key=random.PRNGKey(137), oversampling=10)
print(f"{coll.T} frames, dt = {dt}")
10000 frames, dt = 0.02

coll is a TrajectoryCollection, the data container every SFI tool consumes — whether it comes from a simulation or from a recording (from_arrays(), load()).


First 50 time units of the trajectory

Infer the diffusion and the force

In a real experiment we would not know that the force is linear, so we offer SFI a generic library of candidate terms, the monomials \(\{1, x, x^2\}\), and fit their coefficients by linear regression on the increments.

The data are clean (no measurement noise), so we select the clean-data estimators: "WeakNoise" for \(D\) (it differences successive displacements, which cancels the drift) and preset="clean" for the force. On real recordings keep the defaults ("auto"), which detect localization noise and switch to noise-robust estimators.

from SFI.bases import monomials_up_to

inf = SFI.OverdampedLangevinInference(coll)
inf.compute_diffusion_constant(method="WeakNoise")

B = monomials_up_to(order=2, dim=1, rank="vector")   # candidate terms 1, x, x²
inf.infer_force_linear(B, preset="clean")
inf.compute_force_error()
inf.print_report()
  --- StochasticForceInference Report ---
Average diffusion tensor:
 [[0.4868411]]
Measurement noise tensor:
 [[8.632985e-05]]
Force estimated information: 56.1307258605957
Force: estimated normalized mean squared error (sampling only): 0.026723333721616876
Force Gram condition number: 5.31

  Force Coefficient Table
  ────────────────────────────────────────────────────────────────
  #    Label     Coefficient       Std.Err     SNR  Sig
  ────────────────────────────────────────────────────────────────
  0    1·e0     -6.54218e-02   8.78720e-02     0.7  ·
  1    x0·e0    -1.14331e+00   1.09440e-01    10.4  **
  2    x0^2·e0  -6.37729e-02   1.24944e-01     0.5  ·
  ────────────────────────────────────────────────────────────────
  3/3 basis functions in support, sig: 1* / 1** / 0*** (|SNR| ≥ 2 / 10 / 100)
  (Std.err. reflects sampling error only; discretization bias is not included.)

The report gives each coefficient with its standard error, and a predicted normalized mean-squared error (NMSE) of the whole force — SFI’s own estimate of its accuracy, available without ground truth. The inferred force carries its parameter covariance, so pointwise confidence bands are one call away:

xg = jnp.linspace(-2.5, 2.5, 200)[:, None]
band_full = inf.force_inferred.predict_ci(xg)          # 95% band: "lower", "upper"
Force, full basis (data histogram behind), Coefficients

The 95% error bars cover the truth: the linear coefficient is compatible with \(-k\), the constant and quadratic terms with zero.

Which terms does the data support?

Rather than read the bars by eye, let SFI choose. sparsify_force() searches over subsets of the basis and scores each with an information criterion — by default PASTIS, which admits a spurious term only with small probability.

inf.sparsify_force(criterion="PASTIS")
inf.compute_force_error()
inf.print_report()
band_sparse = inf.force_inferred.predict_ci(xg)
  --- StochasticForceInference Report ---
Average diffusion tensor:
 [[0.4868411]]
Measurement noise tensor:
 [[8.632985e-05]]
Force estimated information: 55.136844635009766
Force: estimated normalized mean squared error (sampling only): 0.009068345740630158
Force Gram condition number: 5.31

  Force Coefficient Table
  ────────────────────────────────────────────────────────────────
  #    Label     Coefficient       Std.Err     SNR  Sig
  ────────────────────────────────────────────────────────────────
  1    x0·e0    -1.11490e+00   1.06170e-01    10.5  **
  ────────────────────────────────────────────────────────────────
  1/3 basis functions in support, sig: 1* / 1** / 0*** (|SNR| ≥ 2 / 10 / 100)
  (Std.err. reflects sampling error only; discretization bias is not included.)
  Zeroed (2): 1·e0, x0^2·e0

The Pareto front (left) shows the decision: the information gained by the best model of each size. The linear term is a huge jump; the other two add next to nothing, and PASTIS (marked) stops at one term. Dropping the spurious terms also tightens the force estimate — compare the band on the right with the full-basis one above.


Pareto front, Force after model selection

Is the fit consistent with the data?

With real data there is no exact model to compare against, so the first check is statistical. diagnose() whitens the residuals, \(z_t = (\Delta x_t - \hat F(x_t)\,\Delta t)/\sqrt{2\hat D\,\Delta t}\), tests whether they look like independent \(\mathcal N(0,1)\) draws, and compares the χ² excess of the residuals with the predicted NMSE.

report = inf.diagnose()
report.print_summary()
Q–Q plot, Residual histogram, Residual ACF
=== SFI diagnostics report ===
backend  : linear
regime   : OD
n_obs    : 9999  n_particles: 1  d: 1
level    : standard

-- Residuals --
  mean = -0.0130  std = 0.9922  skew = -0.023  kurt-3 = +0.004  (n=9999)
  ✓ normality ks                 stat=0.00898   p=0.393
  ✓ autocorr  ljung_box          stat=30.8      p=0.058
  ✓ autocorr  ljung_box_squared  stat=20.6      p=0.424
  predicted NMSE = 0.00907  χ²-excess NMSE = 0.62 (≤ 2.1 at 95%)  χ² z = +0.69 (|z|>5 ⇒ bias)

-- Finite sampling interval --
  dt*|grad F| rms = 0.0223  p99 = 0.0223  (w.r.t. x, 10000 points)  verdict: fine

-- Flags --
  (no issues at α = 0.01)

Mean ≈ 0, standard deviation ≈ 1, no autocorrelation (which would point to missing dynamics) and Gaussian tails: the residuals show no structure the model misses, and their χ² excess is far from the \(|z| > 5\) alarm level. That last test catches gross force biases; at fine sampling its bound on the NMSE is loose, so the predicted NMSE remains the accuracy to quote. The Diagnostics user guide lists every test; Diagnostics — should we trust a fit? shows them catching a wrong model.

Does the model reproduce the data? Bootstrap

A dynamical check: simulate a new trajectory from the inferred model — the fitted force and diffusion are a ready-made simulator — and compare it with the recording. If the fit is faithful the two are statistically indistinguishable. In the movie the recording (top) and a bootstrap (bottom) scroll side by side; on the right each bead rides in the inferred trap \(\hat U(x) = -\int \hat F\,\mathrm{d}x\).

coll_boot, proc_boot = inf.simulate_bootstrapped_trajectory(random.PRNGKey(0))

The sharper test uses statistics the fit never looked at. SFI only uses one-step increments; the decay of correlations over many steps — the autocorrelation \(C(\tau)=\langle x(t)\,x(t+\tau)\rangle\) and the mean-squared displacement \(\mathrm{MSD}(\tau)=\langle[x(t+\tau)-x(t)]^2\rangle\) — is a genuine prediction of the inferred model. Twenty bootstrap trajectories of the same length as the data give the spread these statistics would have if the inferred model were true; the data should fall inside it.

n_boot = 20
boots = [proc_boot.simulate(dt=dt, Nsteps=coll.T, key=random.PRNGKey(s)) for s in range(1, n_boot + 1)]
Autocorrelation, Mean-squared displacement

The data stay inside the bootstrap range at all lags: the short-time force and diffusion reproduce the long-time behaviour.

Save and load

save_results() writes a lightweight summary (coefficients, errors, metadata); save_model() writes the fitted force itself, reloaded against a template that supplies its structure.

import os
import tempfile

from SFI.inference import load_model, save_model, save_results

tmp = tempfile.mkdtemp()
save_results(inf, os.path.join(tmp, "ou_inference"))
save_model(inf.force_inferred, os.path.join(tmp, "ou_force"))
force_reloaded = load_model(os.path.join(tmp, "ou_force"), template=inf.force_inferred)

max_diff = float(jnp.max(jnp.abs(force_reloaded(xg) - inf.force_inferred(xg))))
print(f"Reloaded model matches the original: max |ΔF| = {max_diff:.1e}")
Reloaded model matches the original: max |ΔF| = 0.0e+00

Next steps

Thumbnail

Total running time of the script: (1 minutes 41.999 seconds)

🏷 Tags: synthetic, overdamped, linear, 1D

Gallery generated by Sphinx-Gallery