Fix LFP transients; add baseline drift and optional line noise - #6
Merged
Conversation
The Kasdin all-pole noise filter degenerates to a pure integrator (y[n] = x[n] + y[n-1], pole on the unit circle) at beta == 2. Clipping beta to [0, 2] let sustained high velocity pin beta at that ceiling, so the filter's delay-line state did an unbounded random walk; a subsequent beta change released the accumulated state as a large transient followed by ringing. Capping beta at 1.95 keeps the pole inside the unit circle, eliminating the artifact (worst-case transient dropped ~30x in testing) and removing steady-state drift when velocity saturates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dynamic colored-noise filter runs at the full output rate, so its 1/f slope only holds down to a corner of a few hundred Hz; below that it is white, leaving the baseline flat with no low-frequency wander at rest. Add a BaselineDrift transformer that generates pink (beta=1) noise on a low internal grid (drift_fs, default 50 Hz) where a modest pole count puts the 1/f corner below ~0.3 Hz, then linearly interpolates it up to the output rate and adds it per channel. Drift is velocity-independent, so sub-Hz baseline wander is present even at rest. Filter state, interpolation anchors, and phase are carried across chunks (output is chunk-independent). Wired into Velocity2LFP after mixing and exposed as drift_scale/drift_fs on Velocity2LFPSettings and drift_scale on VelocityEncoderSettings (default scale 8.0 ~= 15% of per-channel LFP std; set 0 to disable). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Line noise (mains pickup) is a common ecephys artifact whose peak frequency slowly wanders as the recording clock drifts relative to the mains. Add: - SinGenerator: optional bounded frequency drift (freq_drift_rate in Hz/min, freq_drift_bound in Hz). When enabled the generator switches to phase accumulation so the waveform stays continuous as the frequency wanders; the frequency offset does a bounded random walk. The no-drift path is unchanged (exact closed form). Core drift + phase-accumulation factored into reusable helpers (freq_drift_step_std, advance_drifting_sine). - LineNoise: additive transformer that adds a single drifting 50/60 Hz sinusoid to every channel, reusing those helpers. freq=None is a pass-through (disabled). State (phase, frequency offset) is carried across chunks for continuity. - VelocityEncoder: LineNoise stage after the spike+LFP sum, exposed via line_noise_freq (None/50/60, default disabled), line_noise_amp, and drift rate/bound. Default drift 0.1 Hz/min, bounded +/-1.5 Hz. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Express the frequency-drift rate in Hz per second instead of Hz per minute. The per-sample random-walk std becomes drift_rate * sqrt(dt), so the RMS wander over 1 s equals drift_rate. Default line-noise drift is now 0.002 Hz/s (equivalent to the previous ~0.1 Hz/min). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pass The drift stage previously wrote the (n_samples, n_channels) array three times (numba interp loop, then `*= scale`, then `data + drift`). Fuse all three into a single numba kernel that writes the result once, with the per-row interpolation weights hoisted out of the channel loop so it compiles to a vectorisable axpy (fastmath enabled). Output is statistically identical (same white-noise draw count and order; continuity/spectrum/reproducibility tests unchanged). At 256 channels this cuts the stage from ~0.31 ms to ~0.05 ms per 30 ms chunk (~6x); the win grows with channel count, which matters for the Raspberry Pi target and planned channel expansion. Warm-up grid generation stays in plain NumPy (runs once at reset). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move BASELINE_DRIFT ahead of MIX_NOISE so the slow 1/f wander is added to the n_lfp_sources pink-noise sources and then spread across output channels by the mixing matrix. This models a shared low-frequency field drift rather than independent per-electrode drift -- good enough for our purposes -- and makes the drift cost scale with n_lfp_sources instead of output_ch (it no longer grows with channel count). Recalibrate the default drift_scale 8.0 -> 4.0 so the mixing-matrix gain (~2.1x) brings the output drift back to ~20% of per-channel LFP std, matching the previous post-mix level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rning The mixing matrix factory was a local closure inside Velocity2LFP.configure, which ezmsg cannot pickle for its settings metadata snapshot (logged as "Could not pickle settings for metadata: Can't get local object 'Velocity2LFP.configure.<locals>.make_mixing_weights'"). Move it to a module-level function bound with functools.partial, which pickles cleanly. Weights are byte-identical to before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes rare large-amplitude transient artifacts in the LFP output and adds two optional, physiologically-motivated artifact sources (slow baseline drift and mains line noise) to the velocity-encoded ecephys simulator. Also includes a performance pass on the drift generator for low-power (Raspberry Pi) targets.
All new behavior is opt-in or matched to prior defaults where it affects existing pipelines.
Changes
1. Fix LFP transient artifacts (β-stability)
The Kasdin all-pole noise filter degenerates to a pure integrator (
y[n] = x[n] + y[n-1], pole on the unit circle) at β = 2. Clipping β to[0, 2]let sustained high velocity pin β at that ceiling, so the filter's state random-walked without bound; a later β change released it as a large transient followed by ringing. Capping β at 1.95 keeps the pole inside the unit circle. In testing this cut the worst-case transient ~30× and removed steady-state drift when velocity saturates.2. Always-on slow 1/f baseline drift
The output-rate Kasdin filter only holds a 1/f slope down to a corner of a few hundred Hz, so the baseline is flat at rest (no sub-Hz power). New
BaselineDrifttransformer generates pink (β=1) noise on a low internal grid (drift_fs, default 50 Hz) — where a modest pole count reaches sub-Hz — then interpolates it up and adds it. It's velocity-independent, so slow wander is present even at rest. Wired intoVelocity2LFPbefore mixing, so it appears as a shared low-frequency field drift across channels and its cost scales withn_lfp_sourcesrather thanoutput_ch.3. Optional mains line noise with clock drift
New
LineNoisetransformer adds a 50/60 Hz sinusoid to all channels, with the peak frequency doing a slow bounded random walk (default 0.002 Hz/s, clamped ±1.5 Hz) to emulate recording-clock drift.freq=Noneis a pass-through (disabled by default).SinGeneratorgains matching optional bounded frequency drift via phase accumulation (the no-drift path stays bit-exact); the drift + phase-accumulation core is shared between the two.4. Performance: fused BaselineDrift kernel
Interpolation, scaling, and the add are fused into a single compiled pass that writes the
(n_samples, n_channels)result once (was three passes), with a SIMD-friendly inner loop. ~6× faster at 256 channels; output is statistically identical (same RNG draw order).Integration notes / defaults
line_noise_freq=None) — set50.0or60.0to enable.drift_scaledefault is tuned so drift is ~20% of per-channel LFP std, matching the level before the pre-mix reorder; set0to disable.VelocityEncoderSettings(drift_scale,line_noise_freq,line_noise_amp,line_noise_drift_rate,line_noise_drift_bound).Testing
pytest: 138 passed, ruff clean. New coverage:test_baseline_drift.py,test_line_noise.py,SinGeneratordrift tests, and a regression test locking in the β<2 stability boundary. Verified continuity/chunk-independence, 1/f spectrum with real sub-Hz power, bounded drift rate/range, and end-to-endVelocityEncoderconfiguration.🤖 Generated with Claude Code