Note
Go to the end to download the full example code.
Getting started: end-to-end inference (Ornstein–Uhlenbeck)¶
The recommended starting point: the complete Stochastic Force Inference (SFI) workflow on the simplest non-trivial system, a particle in a harmonic trap. Its position obeys the overdamped Langevin (Ornstein–Uhlenbeck) equation
a restoring force \(F(x) = -k\,x\) plus white noise of strength \(D\). We simulate a trajectory, then pretend not to know \(F\) or \(D\) and recover them from the data alone:
estimate \(D\) and fit \(F\) on a deliberately over-complete set of candidate terms, with error bars;
let SFI select the minimal model;
validate the fit as one would on real data — residual tests and a bootstrapped trajectory;
save the model and load it back.
Real recordings follow the same path: load your trajectories instead of simulating them (Experimental-data workflow template).
Tags
synthetic · overdamped · linear · 1D
stamp_output()
[Generated: 2026-10-17 23:14]
The data¶
We build the exact model with
OverdampedProcess — the force -k * X(dim=1)
is \(-k\,x\), with X() the coordinate — and record
10 000 frames, i.e. 200 relaxation times \(1/k\).
import SFI
from SFI.bases import X
from SFI.langevin import OverdampedProcess
k_true, D_true = 1.0, 0.5 # trap stiffness and diffusion constant
dt = 0.02 # time between recorded frames
proc = OverdampedProcess(F=-k_true * X(dim=1), D=D_true)
proc.initialize(jnp.array([0.0]))
coll = proc.simulate(dt=dt, Nsteps=10_000, key=random.PRNGKey(137), oversampling=10)
print(f"{coll.T} frames, dt = {dt}")
10000 frames, dt = 0.02
coll is a TrajectoryCollection, the data
container every SFI tool consumes — whether it comes from a simulation
or from a recording (from_arrays(),
load()).
Infer the diffusion and the force¶
In a real experiment we would not know that the force is linear, so we offer SFI a generic library of candidate terms, the monomials \(\{1, x, x^2\}\), and fit their coefficients by linear regression on the increments.
The data are clean (no measurement noise), so we select the clean-data
estimators: "WeakNoise" for \(D\) (it differences successive
displacements, which cancels the drift) and preset="clean" for the
force. On real recordings keep the defaults ("auto"), which detect
localization noise and switch to noise-robust estimators.
from SFI.bases import monomials_up_to
inf = SFI.OverdampedLangevinInference(coll)
inf.compute_diffusion_constant(method="WeakNoise")
B = monomials_up_to(order=2, dim=1, rank="vector") # candidate terms 1, x, x²
inf.infer_force_linear(B, preset="clean")
inf.compute_force_error()
inf.print_report()
--- StochasticForceInference Report ---
Average diffusion tensor:
[[0.4868411]]
Measurement noise tensor:
[[8.632985e-05]]
Force estimated information: 56.1307258605957
Force: estimated normalized mean squared error (sampling only): 0.026723333721616876
Force Gram condition number: 5.31
Force Coefficient Table
────────────────────────────────────────────────────────────────
# Label Coefficient Std.Err SNR Sig
────────────────────────────────────────────────────────────────
0 1·e0 -6.54218e-02 8.78720e-02 0.7 ·
1 x0·e0 -1.14331e+00 1.09440e-01 10.4 **
2 x0^2·e0 -6.37729e-02 1.24944e-01 0.5 ·
────────────────────────────────────────────────────────────────
3/3 basis functions in support, sig: 1* / 1** / 0*** (|SNR| ≥ 2 / 10 / 100)
(Std.err. reflects sampling error only; discretization bias is not included.)
The report gives each coefficient with its standard error, and a predicted normalized mean-squared error (NMSE) of the whole force — SFI’s own estimate of its accuracy, available without ground truth. The inferred force carries its parameter covariance, so pointwise confidence bands are one call away:
xg = jnp.linspace(-2.5, 2.5, 200)[:, None]
band_full = inf.force_inferred.predict_ci(xg) # 95% band: "lower", "upper"
The 95% error bars cover the truth: the linear coefficient is compatible with \(-k\), the constant and quadratic terms with zero.
Which terms does the data support?¶
Rather than read the bars by eye, let SFI choose.
sparsify_force() searches over
subsets of the basis and scores each with an information criterion —
by default PASTIS, which admits a spurious term only with small
probability.
inf.sparsify_force(criterion="PASTIS")
inf.compute_force_error()
inf.print_report()
band_sparse = inf.force_inferred.predict_ci(xg)
--- StochasticForceInference Report ---
Average diffusion tensor:
[[0.4868411]]
Measurement noise tensor:
[[8.632985e-05]]
Force estimated information: 55.136844635009766
Force: estimated normalized mean squared error (sampling only): 0.009068345740630158
Force Gram condition number: 5.31
Force Coefficient Table
────────────────────────────────────────────────────────────────
# Label Coefficient Std.Err SNR Sig
────────────────────────────────────────────────────────────────
1 x0·e0 -1.11490e+00 1.06170e-01 10.5 **
────────────────────────────────────────────────────────────────
1/3 basis functions in support, sig: 1* / 1** / 0*** (|SNR| ≥ 2 / 10 / 100)
(Std.err. reflects sampling error only; discretization bias is not included.)
Zeroed (2): 1·e0, x0^2·e0
The Pareto front (left) shows the decision: the information gained by the best model of each size. The linear term is a huge jump; the other two add next to nothing, and PASTIS (marked) stops at one term. Dropping the spurious terms also tightens the force estimate — compare the band on the right with the full-basis one above.
Is the fit consistent with the data?¶
With real data there is no exact model to compare against, so the
first check is statistical. diagnose()
whitens the residuals,
\(z_t = (\Delta x_t - \hat F(x_t)\,\Delta t)/\sqrt{2\hat D\,\Delta t}\),
tests whether they look like independent \(\mathcal N(0,1)\) draws,
and compares the χ² excess of the residuals with the predicted NMSE.
report = inf.diagnose()
report.print_summary()
=== SFI diagnostics report ===
backend : linear
regime : OD
n_obs : 9999 n_particles: 1 d: 1
level : standard
-- Residuals --
mean = -0.0130 std = 0.9922 skew = -0.023 kurt-3 = +0.004 (n=9999)
✓ normality ks stat=0.00898 p=0.393
✓ autocorr ljung_box stat=30.8 p=0.058
✓ autocorr ljung_box_squared stat=20.6 p=0.424
predicted NMSE = 0.00907 χ²-excess NMSE = 0.62 (≤ 2.1 at 95%) χ² z = +0.69 (|z|>5 ⇒ bias)
-- Finite sampling interval --
dt*|grad F| rms = 0.0223 p99 = 0.0223 (w.r.t. x, 10000 points) verdict: fine
-- Flags --
(no issues at α = 0.01)
Mean ≈ 0, standard deviation ≈ 1, no autocorrelation (which would point to missing dynamics) and Gaussian tails: the residuals show no structure the model misses, and their χ² excess is far from the \(|z| > 5\) alarm level. That last test catches gross force biases; at fine sampling its bound on the NMSE is loose, so the predicted NMSE remains the accuracy to quote. The Diagnostics user guide lists every test; Diagnostics — should we trust a fit? shows them catching a wrong model.
Does the model reproduce the data? Bootstrap¶
A dynamical check: simulate a new trajectory from the inferred model — the fitted force and diffusion are a ready-made simulator — and compare it with the recording. If the fit is faithful the two are statistically indistinguishable. In the movie the recording (top) and a bootstrap (bottom) scroll side by side; on the right each bead rides in the inferred trap \(\hat U(x) = -\int \hat F\,\mathrm{d}x\).
coll_boot, proc_boot = inf.simulate_bootstrapped_trajectory(random.PRNGKey(0))
The sharper test uses statistics the fit never looked at. SFI only uses one-step increments; the decay of correlations over many steps — the autocorrelation \(C(\tau)=\langle x(t)\,x(t+\tau)\rangle\) and the mean-squared displacement \(\mathrm{MSD}(\tau)=\langle[x(t+\tau)-x(t)]^2\rangle\) — is a genuine prediction of the inferred model. Twenty bootstrap trajectories of the same length as the data give the spread these statistics would have if the inferred model were true; the data should fall inside it.
n_boot = 20
boots = [proc_boot.simulate(dt=dt, Nsteps=coll.T, key=random.PRNGKey(s)) for s in range(1, n_boot + 1)]
The data stay inside the bootstrap range at all lags: the short-time force and diffusion reproduce the long-time behaviour.
Save and load¶
save_results() writes a lightweight summary
(coefficients, errors, metadata); save_model()
writes the fitted force itself, reloaded against a template that
supplies its structure.
import os
import tempfile
from SFI.inference import load_model, save_model, save_results
tmp = tempfile.mkdtemp()
save_results(inf, os.path.join(tmp, "ou_inference"))
save_model(inf.force_inferred, os.path.join(tmp, "ou_force"))
force_reloaded = load_model(os.path.join(tmp, "ou_force"), template=inf.force_inferred)
max_diff = float(jnp.max(jnp.abs(force_reloaded(xg) - inf.force_inferred(xg))))
print(f"Reloaded model matches the original: max |ΔF| = {max_diff:.1e}")
Reloaded model matches the original: max |ΔF| = 0.0e+00
Next steps¶
Your own data: Start here: is SFI right for my data? routes you to the right estimator; Experimental-data workflow template is the template for a noisy recording, validated without ground truth.
Noisy or coarsely sampled recordings: the parametric estimators, Measurement noise and coarse sampling.
Building models (custom bases, interactions, masking): Building bases and Models and state functions.
Loading data (CSV/Parquet/HDF5, several experiments): Trajectory data.
Inference in depth (overdamped vs underdamped, diffusion estimators, error analysis): Running inference.
Thumbnail¶
Total running time of the script: (1 minutes 41.999 seconds)