Skip to content

Fix LFP transients; add baseline drift and optional line noise - #6

Merged
cboulay merged 7 commits into
devfrom
fix-lfp-beta-stability
Jul 23, 2026
Merged

Fix LFP transients; add baseline drift and optional line noise#6
cboulay merged 7 commits into
devfrom
fix-lfp-beta-stability

Conversation

@cboulay

@cboulay cboulay commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

Fixes rare large-amplitude transient artifacts in the LFP output and adds two optional, physiologically-motivated artifact sources (slow baseline drift and mains line noise) to the velocity-encoded ecephys simulator. Also includes a performance pass on the drift generator for low-power (Raspberry Pi) targets.

All new behavior is opt-in or matched to prior defaults where it affects existing pipelines.

Changes

1. Fix LFP transient artifacts (β-stability)
The Kasdin all-pole noise filter degenerates to a pure integrator (y[n] = x[n] + y[n-1], pole on the unit circle) at β = 2. Clipping β to [0, 2] let sustained high velocity pin β at that ceiling, so the filter's state random-walked without bound; a later β change released it as a large transient followed by ringing. Capping β at 1.95 keeps the pole inside the unit circle. In testing this cut the worst-case transient ~30× and removed steady-state drift when velocity saturates.

2. Always-on slow 1/f baseline drift
The output-rate Kasdin filter only holds a 1/f slope down to a corner of a few hundred Hz, so the baseline is flat at rest (no sub-Hz power). New BaselineDrift transformer generates pink (β=1) noise on a low internal grid (drift_fs, default 50 Hz) — where a modest pole count reaches sub-Hz — then interpolates it up and adds it. It's velocity-independent, so slow wander is present even at rest. Wired into Velocity2LFP before mixing, so it appears as a shared low-frequency field drift across channels and its cost scales with n_lfp_sources rather than output_ch.

3. Optional mains line noise with clock drift
New LineNoise transformer adds a 50/60 Hz sinusoid to all channels, with the peak frequency doing a slow bounded random walk (default 0.002 Hz/s, clamped ±1.5 Hz) to emulate recording-clock drift. freq=None is a pass-through (disabled by default). SinGenerator gains matching optional bounded frequency drift via phase accumulation (the no-drift path stays bit-exact); the drift + phase-accumulation core is shared between the two.

4. Performance: fused BaselineDrift kernel
Interpolation, scaling, and the add are fused into a single compiled pass that writes the (n_samples, n_channels) result once (was three passes), with a SIMD-friendly inner loop. ~6× faster at 256 channels; output is statistically identical (same RNG draw order).

Integration notes / defaults

  • Line noise is off by default (line_noise_freq=None) — set 50.0 or 60.0 to enable.
  • drift_scale default is tuned so drift is ~20% of per-channel LFP std, matching the level before the pre-mix reorder; set 0 to disable.
  • These are exposed on VelocityEncoderSettings (drift_scale, line_noise_freq, line_noise_amp, line_noise_drift_rate, line_noise_drift_bound).

Testing

pytest: 138 passed, ruff clean. New coverage: test_baseline_drift.py, test_line_noise.py, SinGenerator drift tests, and a regression test locking in the β<2 stability boundary. Verified continuity/chunk-independence, 1/f spectrum with real sub-Hz power, bounded drift rate/range, and end-to-end VelocityEncoder configuration.

🤖 Generated with Claude Code

cboulay and others added 7 commits July 23, 2026 09:01
The Kasdin all-pole noise filter degenerates to a pure integrator
(y[n] = x[n] + y[n-1], pole on the unit circle) at beta == 2. Clipping
beta to [0, 2] let sustained high velocity pin beta at that ceiling, so
the filter's delay-line state did an unbounded random walk; a subsequent
beta change released the accumulated state as a large transient followed
by ringing. Capping beta at 1.95 keeps the pole inside the unit circle,
eliminating the artifact (worst-case transient dropped ~30x in testing)
and removing steady-state drift when velocity saturates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dynamic colored-noise filter runs at the full output rate, so its 1/f
slope only holds down to a corner of a few hundred Hz; below that it is
white, leaving the baseline flat with no low-frequency wander at rest.

Add a BaselineDrift transformer that generates pink (beta=1) noise on a
low internal grid (drift_fs, default 50 Hz) where a modest pole count puts
the 1/f corner below ~0.3 Hz, then linearly interpolates it up to the
output rate and adds it per channel. Drift is velocity-independent, so
sub-Hz baseline wander is present even at rest. Filter state, interpolation
anchors, and phase are carried across chunks (output is chunk-independent).

Wired into Velocity2LFP after mixing and exposed as drift_scale/drift_fs on
Velocity2LFPSettings and drift_scale on VelocityEncoderSettings (default
scale 8.0 ~= 15% of per-channel LFP std; set 0 to disable).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Line noise (mains pickup) is a common ecephys artifact whose peak frequency
slowly wanders as the recording clock drifts relative to the mains. Add:

- SinGenerator: optional bounded frequency drift (freq_drift_rate in Hz/min,
  freq_drift_bound in Hz). When enabled the generator switches to phase
  accumulation so the waveform stays continuous as the frequency wanders; the
  frequency offset does a bounded random walk. The no-drift path is unchanged
  (exact closed form). Core drift + phase-accumulation factored into reusable
  helpers (freq_drift_step_std, advance_drifting_sine).

- LineNoise: additive transformer that adds a single drifting 50/60 Hz sinusoid
  to every channel, reusing those helpers. freq=None is a pass-through (disabled).
  State (phase, frequency offset) is carried across chunks for continuity.

- VelocityEncoder: LineNoise stage after the spike+LFP sum, exposed via
  line_noise_freq (None/50/60, default disabled), line_noise_amp, and drift
  rate/bound. Default drift 0.1 Hz/min, bounded +/-1.5 Hz.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Express the frequency-drift rate in Hz per second instead of Hz per minute.
The per-sample random-walk std becomes drift_rate * sqrt(dt), so the RMS
wander over 1 s equals drift_rate. Default line-noise drift is now
0.002 Hz/s (equivalent to the previous ~0.1 Hz/min).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pass

The drift stage previously wrote the (n_samples, n_channels) array three times
(numba interp loop, then `*= scale`, then `data + drift`). Fuse all three into a
single numba kernel that writes the result once, with the per-row interpolation
weights hoisted out of the channel loop so it compiles to a vectorisable axpy
(fastmath enabled). Output is statistically identical (same white-noise draw
count and order; continuity/spectrum/reproducibility tests unchanged).

At 256 channels this cuts the stage from ~0.31 ms to ~0.05 ms per 30 ms chunk
(~6x); the win grows with channel count, which matters for the Raspberry Pi
target and planned channel expansion. Warm-up grid generation stays in plain
NumPy (runs once at reset).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move BASELINE_DRIFT ahead of MIX_NOISE so the slow 1/f wander is added to the
n_lfp_sources pink-noise sources and then spread across output channels by the
mixing matrix. This models a shared low-frequency field drift rather than
independent per-electrode drift -- good enough for our purposes -- and makes the
drift cost scale with n_lfp_sources instead of output_ch (it no longer grows
with channel count).

Recalibrate the default drift_scale 8.0 -> 4.0 so the mixing-matrix gain
(~2.1x) brings the output drift back to ~20% of per-channel LFP std, matching
the previous post-mix level.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rning

The mixing matrix factory was a local closure inside Velocity2LFP.configure,
which ezmsg cannot pickle for its settings metadata snapshot (logged as
"Could not pickle settings for metadata: Can't get local object
'Velocity2LFP.configure.<locals>.make_mixing_weights'"). Move it to a
module-level function bound with functools.partial, which pickles cleanly.
Weights are byte-identical to before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cboulay
cboulay merged commit dbe9e33 into dev Jul 23, 2026
14 checks passed
@cboulay
cboulay deleted the fix-lfp-beta-stability branch July 23, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant