Skip to content

Use probeinterface's renamed Neuropixels readers and new probe detectors #4593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alejoe91 merged 4 commits into from
Jun 25, 2026
Merged

Use probeinterface's renamed Neuropixels readers and new probe detectors #4593

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6fc5d27
use new neurpixels detectors in open ephys and spikegadgets
h-mayorquin May 22, 2026
8ba6d2a
Merge branch 'main' into neuropixels_use_new_probe_extraciton_functions
alejoe91 Jun 8, 2026
e4dacc5
feat: add ADC sample shift and saturation threshold to spikegadgets r…
alejoe91 Jun 11, 2026
7529319
fix: update sample shift logic to use device_channel_indices for inde…
alejoe91 Jun 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions src/spikeinterface/extractors/neoextractors/openephys.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -324,14 +324,12 @@ def __init__(
if "NI-DAQmx" not in stream_name:
settings_file = node_structure["experiments"][exp_id]["settings_file"]

if Path(settings_file).is_file():
probe = probeinterface.read_openephys(
settings_file=settings_file, stream_name=oe_stream_name, raise_error=False
if Path(settings_file).is_file() and probeinterface.has_neuropixels_probes(
settings_file, stream_name=oe_stream_name
):
probe = probeinterface.read_openephys_neuropixels(
settings_file=settings_file, stream_name=oe_stream_name
)
else:
probe = None

if probe is not None:
if probe.shank_ids is not None:
self.set_probe(probe, in_place=True, group_mode="by_shank")
else:
Expand Down
34 changes: 27 additions & 7 deletions src/spikeinterface/extractors/neoextractors/spikegadgets.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
from pathlib import Path
import warnings
import numpy as np

import packaging

import packaging.version
import probeinterface
from spikeinterface.core.core_tools import define_function_from_class
from spikeinterface.extractors.neuropixels_utils import (
get_neuropixels_sample_shifts_from_probe,
compute_saturation_threshold_from_probe,
)

from .neobaseextractor import NeoBaseRecordingExtractor

Expand Down Expand Up @@ -56,12 +59,29 @@ def __init__(
)
self._kwargs.update(dict(file_path=str(Path(file_path).absolute()), stream_id=stream_id))

probegroup = probeinterface.read_spikegadgets(file_path, raise_error=False)
# TODO: add adc sample shifts and saturation levels if available in the probe metadata

if probegroup is not None:
if probeinterface.has_spikegadgets_neuropixels_probes(file_path):
probegroup = probeinterface.read_spikegadgets_neuropixels(file_path)
self.set_probegroup(probegroup, in_place=True)

# get inter-sample shifts based on the probe information and mux channels
sample_shifts = np.array([])
saturation_thresholds_uV = []
for probe in probegroup.probes:
sample_shifts_probe = get_neuropixels_sample_shifts_from_probe(probe)

@h-mayorquin h-mayorquin Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function get_neuropixels_sample_shifts_from_probe returns the sample shifts in probe (contact) order, so before you set them as a channel property you need to re-wire them into channel order. This works for SpikeGLX and OpenEphys without any extra step because probeinterface already wires those probes so their contacts are in the order of the binary traces (device_channel_indices = arange), so contact order already equals channel order.

We could do that re-wiring here too, but it is more complicated than in the SpikeGLX and OpenEphys case. Those write one file per probe, while SpikeGadgets puts more than one probe in a single file. For the dataset with neuropixels that we have on gin already (SpikeGadgets_test_data_2xNpix1.0_20240318_173658.rec, the two probes are interleaved in that single binary in blocks of 32 channels (channels 0-31 are probe 0, 32-63 are probe 1, 64-95 are probe 0, and so on), so a probe's contacts do not map to a contiguous run of channels and device_channel_indices is a real permutation (hwChan) rather than arange. That is why the current np.concatenate of per-probe shifts lands on the wrong channels (704 of 768 on this fixture).

I see three possible paths forward:

  1. Re-route the output of get_neuropixels_sample_shifts_from_probe through device_channel_indices. Each probe's device_channel_indices tells you which recording channel each contact maps to, so you scatter the per-probe shifts into a channel-length array:
sample_shifts = np.zeros(self.get_num_channels())
for probe in probegroup.probes:
    shifts = get_neuropixels_sample_shifts_from_probe(probe)
    if shifts is not None:
        sample_shifts[probe.device_channel_indices] = shifts   # scatter into channel order
self.set_property("inter_sample_shift", sample_shifts)

  1. Re-sort at the probeinterface level using the global device channel indices, sorting each probe/group so the contacts come out in channel order. I am not sure this is possible, since each probe owns a non-contiguous, interleaved set of channels, so a per-probe arange might not be doable but given that you guys like that pattern maybe you might be able to find it. I don't like the pattern and you know how motivated thinking is a horrible beast.

  2. Use the property from the probe after you set it. This does the rearranging after we set it on the recording (so it is already re-ordered by set_probegroup): read self.get_property("contact_vector")["adc_sample_order"], which is already one value per channel in channel order, and set the property from that.

I honestly prefer 1: explicit is better than implicit, and it will not clash with any of the current Probe refactorings like option 3 might. Option 2 will require more work on your side and I would like to merge this and then do that if you want to move that way.

@alejoe91 alejoe91 Jun 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@h-mayorquin thanks! This was super helpful!

I agree option one is the cleanest. Implemented in last commit and fixed the sample_shift logic. Since now we pre-allocate an array, insted of checking the length I instantiate the values as -1, and we check that no -1 are left (since sample shifts are [0-1])

@h-mayorquin h-mayorquin Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I am glad it helped. This is fine to merge to me.

if sample_shifts_probe is not None:
sample_shifts = np.concatenate([sample_shifts, sample_shifts_probe])
# add saturation levels if available
saturation_threshold_uV_probe = compute_saturation_threshold_from_probe(probe, self.stream_id)
if saturation_threshold_uV_probe is not None:
saturation_thresholds_uV.append(saturation_threshold_uV_probe)

if len(sample_shifts) > self.get_num_channels():

@h-mayorquin h-mayorquin Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be equal?

@alejoe91 alejoe91 Jun 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oups yes of course! Good catch!

@alejoe91 alejoe91 Jun 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in last commit.

self.set_property("inter_sample_shift", sample_shifts)
if len(set(saturation_thresholds_uV)) == 1:
self.annotate(saturation_threshold_uV=saturation_thresholds_uV[0])
else:
warnings.warn("Multiple saturation thresholds found for different probes, unable to annotate.")

@classmethod
def map_to_neo_kwargs(cls, file_path):
neo_kwargs = {"filename": str(file_path)}
Expand Down
Loading