Skill Tracer
Skills › Scientific Agent Skills › neuropixels-analysis › PREPROCESSING.md

PREPROCESSING.md

92 lines · skills/neuropixels-analysis/references/PREPROCESSING.md at 5f2bc03

Open on GitHub
1# Neuropixels preprocessing
2
3Targets SpikeInterface 0.105.0, ProbeInterface 0.4.0 and Neo 0.14.5; reviewed
42026-10-01. Real acquisition examples require validation on the supplied files.
5

Neuropixels preprocessing

Targets SpikeInterface 0.105.0, ProbeInterface 0.4.0 and Neo 0.14.5; reviewed 2026-10-01. Real acquisition examples require validation on the supplied files.

Raw lines 1 to 5
6## Load the correct stream and preserve its metadata
7
8```python
9import spikeinterface.full as si
10names, ids = si.get_neo_streams("spikeglx", "raw_data/")
11print(names, ids)
12recording = si.read_spikeglx("raw_data/", stream_name="imec0.ap") # Replace with the inspected AP name.
13```
15Select the inspected AP stream; never assume the first stream, `imec0.ap`, or a
16384-channel count is correct for every acquisition. In 0.105, sync is a separate
17stream and `read_spikeglx` no longer accepts `load_sync_channel`. Retain the `.meta`
18file and check gain/offset, sampling rate, contact positions, shank IDs, channel
19order, sample timing, and selected bank. Do not treat sync or auxiliary channels
20as electrode signals. ProbeInterface 0.4 uses probe part numbers to obtain geometry.
21For Open Ephys, inspect streams using `get_neo_streams("openephys", folder)` and
22pass `stream_name` explicitly. For NWB, select the intended `electrical_series_path`.
24`get_traces()` returns **samples × channels**, in stored units by default.
25`return_in_uV=True` requires calibrated `gain_to_uV` and `offset_to_uV`. Geometry
26is in micrometers. Channel IDs are labels, not array indices. Never set a gain of
271 merely to bypass calibration checks. Preserve discontinuities and time origins;
28the bundled commands require one continuous segment and a single attached probe.
29Referencing requires explicit shank IDs; the pipeline motion stage further requires
30one shank. Split multishank recordings explicitly for separate motion estimates.

Load the correct stream and preserve its metadata

import spikeinterface.full as si
names, ids = si.get_neo_streams("spikeglx", "raw_data/")
print(names, ids)
recording = si.read_spikeglx("raw_data/", stream_name="imec0.ap")  # Replace with the inspected AP name.

Select the inspected AP stream; never assume the first stream, imec0.ap, or a 384-channel count is correct for every acquisition. In 0.105, sync is a separate stream and read_spikeglx no longer accepts load_sync_channel. Retain the .meta file and check gain/offset, sampling rate, contact positions, shank IDs, channel order, sample timing, and selected bank. Do not treat sync or auxiliary channels as electrode signals. ProbeInterface 0.4 uses probe part numbers to obtain geometry. For Open Ephys, inspect streams using get_neo_streams("openephys", folder) and pass stream_name explicitly. For NWB, select the intended electrical_series_path.

get_traces() returns samples × channels, in stored units by default. return_in_uV=True requires calibrated gain_to_uV and offset_to_uV. Geometry is in micrometers. Channel IDs are labels, not array indices. Never set a gain of 1 merely to bypass calibration checks. Preserve discontinuities and time origins; the bundled commands require one continuous segment and a single attached probe. Referencing requires explicit shank IDs; the pipeline motion stage further requires one shank. Split multishank recordings explicitly for separate motion estimates.

Raw lines 6 to 31
32## Filter, remove bad channels, correct timing, reference per shank
34```python
35# From the skill root; these are the same checks used by the CLI.
36import sys
37sys.path.insert(0, "scripts")
38from _common import validate_recording, apply_phase_correction, reference_by_shank
40validate_recording(recording)
41rec = si.highpass_filter(recording, freq_min=400.0)
42bad_ids, labels = si.detect_bad_channels(rec, seed=42)
43print(dict(zip(rec.channel_ids, labels))) # labels cover ALL original channels
44rec = rec.remove_channels(bad_ids)
45rec = apply_phase_correction(rec)
46rec = reference_by_shank(rec)
47rec = rec.save(folder="preprocessed/", format="binary", n_jobs=1)
48```
50Apply phase correction from acquisition `inter_sample_shift` metadata, not a rule
51that NP2 has one ADC. If metadata is absent, verify the probe/acquisition timing
52and explicitly disable the step when appropriate. Do not fabricate timing offsets.
53The helper fails rather than guessing. Removing channels retains their mapping.
54Common reference `groups` expects **lists of channel IDs**, not one group label per
55channel. Reference separate shanks independently; never assume `group` means shank.
57A bandpass alternative is `si.bandpass_filter(rec, freq_min=300, freq_max=6000)`;
58require `0 < freq_min < freq_max < fs / 2`. Use default filter margins unless
59validated otherwise. AP filtering removes low frequencies; keep the original LF
60stream for LFP analysis. Notch filtering at 50/60 Hz is usually irrelevant after
61an AP highpass and can distort signals when applied indiscriminately.

Filter, remove bad channels, correct timing, reference per shank

# From the skill root; these are the same checks used by the CLI.
import sys
sys.path.insert(0, "scripts")
from _common import validate_recording, apply_phase_correction, reference_by_shank

validate_recording(recording)
rec = si.highpass_filter(recording, freq_min=400.0)
bad_ids, labels = si.detect_bad_channels(rec, seed=42)
print(dict(zip(rec.channel_ids, labels)))  # labels cover ALL original channels
rec = rec.remove_channels(bad_ids)
rec = apply_phase_correction(rec)
rec = reference_by_shank(rec)
rec = rec.save(folder="preprocessed/", format="binary", n_jobs=1)

Apply phase correction from acquisition inter_sample_shift metadata, not a rule that NP2 has one ADC. If metadata is absent, verify the probe/acquisition timing and explicitly disable the step when appropriate. Do not fabricate timing offsets. The helper fails rather than guessing. Removing channels retains their mapping. Common reference groups expects lists of channel IDs, not one group label per channel. Reference separate shanks independently; never assume group means shank.

A bandpass alternative is si.bandpass_filter(rec, freq_min=300, freq_max=6000); require 0 < freq_min < freq_max < fs / 2. Use default filter margins unless validated otherwise. AP filtering removes low frequencies; keep the original LF stream for LFP analysis. Notch filtering at 50/60 Hz is usually irrelevant after an AP highpass and can distort signals when applied indiscriminately.

Raw lines 32 to 62
63## Optional operations
65```python
66# Geometry-based interpolation is an alternative to removing channels.
67# Bad-channel IDs must refer to this recording, before their removal.
68rec_interpolated = si.interpolate_bad_channels(recording, bad_channel_ids=bad_ids)
70# Whitening is sorter-specific; do not use whitened traces for physical amplitudes.
71whitened = si.whiten(rec, mode="local", radius_um=100)
73# Per-segment lists of stimulus sample indices, not seconds.
74artifact_clean = si.remove_artifacts(rec, [[10000, 20000]],
75 ms_before=0.5, ms_after=3.0, mode="linear")
76```
78Interpolation changes neighboring noise statistics and is not always preferable
79to removal. Review stimulation/saturation windows and mask invalid analysis periods;
80there is no `blank_staturation` API. Spatial highpass destriping via
81`si.highpass_spatial_filter` needs a depth-ordered single shank and sufficient
82contacts; a common median reference alone is not full IBL destriping.
84Use [MOTION_CORRECTION.md](MOTION_CORRECTION.md) for motion estimation/interpolation.
85Reload a saved extractor with `si.load("preprocessed/")`. Binary caching is a
86time/disk tradeoff; estimate storage before writing a full recording. The standalone
87preprocessing command puts the extractor in `<output>/preprocessed/`; downstream
88commands accept that folder or its parent.
90Sources: [Neuropixels how-to](https://spikeinterface.readthedocs.io/en/stable/how_to/analyze_neuropixels.html),
91[API](https://spikeinterface.readthedocs.io/en/stable/api.html),
92[ProbeInterface catalogue](https://probeinterface.readthedocs.io/en/main/neuropixels_readers.html).

Optional operations

# Geometry-based interpolation is an alternative to removing channels.
# Bad-channel IDs must refer to this recording, before their removal.
rec_interpolated = si.interpolate_bad_channels(recording, bad_channel_ids=bad_ids)

# Whitening is sorter-specific; do not use whitened traces for physical amplitudes.
whitened = si.whiten(rec, mode="local", radius_um=100)

# Per-segment lists of stimulus sample indices, not seconds.
artifact_clean = si.remove_artifacts(rec, [[10000, 20000]],
                                    ms_before=0.5, ms_after=3.0, mode="linear")

Interpolation changes neighboring noise statistics and is not always preferable to removal. Review stimulation/saturation windows and mask invalid analysis periods; there is no blank_staturation API. Spatial highpass destriping via si.highpass_spatial_filter needs a depth-ordered single shank and sufficient contacts; a common median reference alone is not full IBL destriping.

Use MOTION_CORRECTION.md for motion estimation/interpolation. Reload a saved extractor with si.load("preprocessed/"). Binary caching is a time/disk tradeoff; estimate storage before writing a full recording. The standalone preprocessing command puts the extractor in <output>/preprocessed/; downstream commands accept that folder or its parent.

Sources: Neuropixels how-to, API, ProbeInterface catalogue.

Raw lines 63 to 92