Skill Tracer
Skills › Scientific Agent Skills › neuropixels-analysis › standard_workflow.md

standard_workflow.md

313 lines · skills/neuropixels-analysis/references/standard_workflow.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Standard Neuropixels Analysis Workflow
2
3Complete step-by-step guide for analyzing Neuropixels recordings from raw data to curated
4units, using SpikeInterface 0.105.0 (reviewed 2026-10-01). Real-data/sorter/model
5examples are illustrative; validate acquisition calibration, probe mapping and
6segment time origins before running them.
7

Standard Neuropixels Analysis Workflow

Complete step-by-step guide for analyzing Neuropixels recordings from raw data to curated units, using SpikeInterface 0.105.0 (reviewed 2026-10-01). Real-data/sorter/model examples are illustrative; validate acquisition calibration, probe mapping and segment time origins before running them.

Raw lines 1 to 7
8## Overview
9
10```
11Raw Recording → Preprocessing → Motion Correction → Spike Sorting →
12Postprocessing → Quality Metrics → Curation → Export
13```
15```python
16import spikeinterface.full as si
17import spikeinterface.curation as sc
19si.set_global_job_kwargs(n_jobs=-1, chunk_duration="1s", progress_bar=True)
20```

Overview

Raw Recording → Preprocessing → Motion Correction → Spike Sorting →
Postprocessing → Quality Metrics → Curation → Export
import spikeinterface.full as si
import spikeinterface.curation as sc

si.set_global_job_kwargs(n_jobs=-1, chunk_duration="1s", progress_bar=True)
Raw lines 8 to 21
22## 1. Data Loading

1. Data Loading

Raw lines 22 to 23
24### Supported formats
26```python
27# Inspect streams first
28stream_names, stream_ids = si.get_neo_streams("spikeglx", "/path/to/run_g0/")
30# SpikeGLX (most common)
31recording = si.read_spikeglx("/path/to/run_g0/", stream_name="imec0.ap")
33# Open Ephys
34recording = si.read_openephys("/path/to/experiment/")
36# NWB
37recording = si.read_nwb("/path/to/file.nwb")
38```

Supported formats

# Inspect streams first
stream_names, stream_ids = si.get_neo_streams("spikeglx", "/path/to/run_g0/")

# SpikeGLX (most common)
recording = si.read_spikeglx("/path/to/run_g0/", stream_name="imec0.ap")

# Open Ephys
recording = si.read_openephys("/path/to/experiment/")

# NWB
recording = si.read_nwb("/path/to/file.nwb")
Raw lines 24 to 39
40### Verify recording properties
42```python
43print(f"Channels: {recording.get_num_channels()}")
44print(f"Duration: {recording.get_total_duration():.1f}s")
45print(f"Sampling rate: {recording.get_sampling_frequency()}Hz")
46print(f"Probe: {recording.get_probe()}")
47locations = recording.get_channel_locations()
48```

Verify recording properties

print(f"Channels: {recording.get_num_channels()}")
print(f"Duration: {recording.get_total_duration():.1f}s")
print(f"Sampling rate: {recording.get_sampling_frequency()}Hz")
print(f"Probe: {recording.get_probe()}")
locations = recording.get_channel_locations()
Raw lines 40 to 49
50## 2. Preprocessing

2. Preprocessing

Raw lines 50 to 51
52### Filter/reference chain (single shank; not full IBL destriping)
54```python
55rec = si.highpass_filter(recording, freq_min=400.0)
56bad_channel_ids, channel_labels = si.detect_bad_channels(rec)
57rec = rec.remove_channels(bad_channel_ids)
58rec = si.phase_shift(rec) # Requires inter_sample_shift metadata
59rec = si.common_reference(rec, operator="median", reference="global")
60```
62A bandpass alternative (some labs prefer an explicit passband):
64```python
65rec = si.bandpass_filter(recording, freq_min=300.0, freq_max=6000.0)
66rec = si.phase_shift(rec)
67bad_channel_ids, _ = si.detect_bad_channels(rec)
68rec = rec.remove_channels(bad_channel_ids)
69rec = si.common_reference(rec, operator="median", reference="global")
70```

Filter/reference chain (single shank; not full IBL destriping)

rec = si.highpass_filter(recording, freq_min=400.0)
bad_channel_ids, channel_labels = si.detect_bad_channels(rec)
rec = rec.remove_channels(bad_channel_ids)
rec = si.phase_shift(rec)                                   # Requires inter_sample_shift metadata
rec = si.common_reference(rec, operator="median", reference="global")

A bandpass alternative (some labs prefer an explicit passband):

rec = si.bandpass_filter(recording, freq_min=300.0, freq_max=6000.0)
rec = si.phase_shift(rec)
bad_channel_ids, _ = si.detect_bad_channels(rec)
rec = rec.remove_channels(bad_channel_ids)
rec = si.common_reference(rec, operator="median", reference="global")
Raw lines 52 to 71
72### Spatial destriping (strong artifacts)
74```python
75rec = si.highpass_filter(recording, freq_min=400.0)
76rec = si.phase_shift(rec)
77rec = si.highpass_spatial_filter(rec) # destriping
78rec = si.common_reference(rec, operator="median", reference="global")
79```

Spatial destriping (strong artifacts)

rec = si.highpass_filter(recording, freq_min=400.0)
rec = si.phase_shift(rec)
rec = si.highpass_spatial_filter(rec)                       # destriping
rec = si.common_reference(rec, operator="median", reference="global")
Raw lines 72 to 80
81### Save preprocessed data
83```python
84rec = rec.save(folder="preprocessed/", format="binary")
85```

Save preprocessed data

rec = rec.save(folder="preprocessed/", format="binary")
Raw lines 81 to 86
87## 3. Motion/Drift Correction

3. Motion/Drift Correction

Raw lines 87 to 88
89### Check whether correction is needed
91```python
92from spikeinterface.sortingcomponents.peak_detection import detect_peaks
93from spikeinterface.sortingcomponents.peak_localization import localize_peaks
95noise_levels = si.get_noise_levels(rec, return_in_uV=False)
96peaks = detect_peaks(rec, method='locally_exclusive', method_kwargs={'noise_levels': noise_levels, 'detect_threshold': 5, 'radius_um': 50.0})
97peak_locations = localize_peaks(rec, peaks, method="center_of_mass")
99si.plot_drift_raster_map(peaks=peaks, peak_locations=peak_locations, recording=rec, clim=(-50, 50))
100```

Check whether correction is needed

from spikeinterface.sortingcomponents.peak_detection import detect_peaks
from spikeinterface.sortingcomponents.peak_localization import localize_peaks

noise_levels = si.get_noise_levels(rec, return_in_uV=False)
peaks = detect_peaks(rec, method='locally_exclusive', method_kwargs={'noise_levels': noise_levels, 'detect_threshold': 5, 'radius_um': 50.0})
peak_locations = localize_peaks(rec, peaks, method="center_of_mass")

si.plot_drift_raster_map(peaks=peaks, peak_locations=peak_locations, recording=rec, clim=(-50, 50))
Raw lines 89 to 101
102### Apply correction
104```python
105# One-call correction with a preset
106rec_corrected = si.correct_motion(rec, preset="nonrigid_fast_and_accurate", folder="motion/")
107```
109See [MOTION_CORRECTION.md](MOTION_CORRECTION.md) for the full estimate/interpolate pipeline
110and DREDge usage.

Apply correction

# One-call correction with a preset
rec_corrected = si.correct_motion(rec, preset="nonrigid_fast_and_accurate", folder="motion/")

See MOTION_CORRECTION.md for the full estimate/interpolate pipeline and DREDge usage.

Raw lines 102 to 111
112## 4. Spike Sorting

4. Spike Sorting

Raw lines 112 to 113
114### Recommended: Kilosort4
116```python
117sorting = si.run_sorter("kilosort4", rec_corrected, folder="sorting_KS4/", do_correction=False, verbose=True)
119# With custom parameters
120sorting = si.run_sorter(
121 "kilosort4", rec_corrected, folder="sorting_KS4/",
122 do_correction=False, # Input is already motion-corrected.
123 Th_universal=9, # detection threshold
124 Th_learned=8,
125 batch_size=60000,
127```

Recommended: Kilosort4

sorting = si.run_sorter("kilosort4", rec_corrected, folder="sorting_KS4/", do_correction=False, verbose=True)

# With custom parameters
sorting = si.run_sorter(
    "kilosort4", rec_corrected, folder="sorting_KS4/",
    do_correction=False,  # Input is already motion-corrected.
    Th_universal=9,      # detection threshold
    Th_learned=8,
    batch_size=60000,
)
Raw lines 114 to 128
129### Alternative sorters
131```python
132sorting = si.run_sorter("spykingcircus2", rec_corrected, folder="sc2/", apply_motion_correction=False) # CPU
133sorting = si.run_sorter("tridesclous2", rec_corrected, folder="tdc2/") # CPU
134sorting = si.run_sorter("mountainsort5", rec_corrected, folder="ms5/") # CPU
135```

Alternative sorters

sorting = si.run_sorter("spykingcircus2", rec_corrected, folder="sc2/", apply_motion_correction=False)   # CPU
sorting = si.run_sorter("tridesclous2", rec_corrected, folder="tdc2/")    # CPU
sorting = si.run_sorter("mountainsort5", rec_corrected, folder="ms5/")    # CPU
Raw lines 129 to 136
137### Compare multiple sorters
139```python
140sortings = {
141 "kilosort4": si.run_sorter("kilosort4", rec_corrected, folder="ks4/", do_correction=False),
142 "spykingcircus2": si.run_sorter("spykingcircus2", rec_corrected, folder="sc2/", apply_motion_correction=False),
145comparison = si.compare_multiple_sorters(list(sortings.values()),
146 name_list=list(sortings.keys()))
147agreement = comparison.get_agreement_sorting(minimum_agreement_count=2)
148```

Compare multiple sorters

sortings = {
    "kilosort4": si.run_sorter("kilosort4", rec_corrected, folder="ks4/", do_correction=False),
    "spykingcircus2": si.run_sorter("spykingcircus2", rec_corrected, folder="sc2/", apply_motion_correction=False),
}

comparison = si.compare_multiple_sorters(list(sortings.values()),
                                         name_list=list(sortings.keys()))
agreement = comparison.get_agreement_sorting(minimum_agreement_count=2)
Raw lines 137 to 149
150## 5. Postprocessing

5. Postprocessing

Raw lines 150 to 151
152### Create analyzer and compute extensions
154```python
155analyzer = si.create_sorting_analyzer(sorting, rec_corrected, sparse=True,
156 format="binary_folder", folder="analyzer/")
158analyzer.compute("random_spikes", method="uniform", max_spikes_per_unit=500)
159analyzer.compute("waveforms", ms_before=1.0, ms_after=2.0)
160analyzer.compute("templates", operators=["average", "std"])
161analyzer.compute("noise_levels")
162analyzer.compute("spike_amplitudes")
163analyzer.compute("amplitude_scalings")
164analyzer.compute("correlograms", window_ms=50.0, bin_ms=1.0)
165analyzer.compute("unit_locations", method="monopolar_triangulation")
166analyzer.compute("template_similarity")
167```

Create analyzer and compute extensions

analyzer = si.create_sorting_analyzer(sorting, rec_corrected, sparse=True,
                                      format="binary_folder", folder="analyzer/")

analyzer.compute("random_spikes", method="uniform", max_spikes_per_unit=500)
analyzer.compute("waveforms", ms_before=1.0, ms_after=2.0)
analyzer.compute("templates", operators=["average", "std"])
analyzer.compute("noise_levels")
analyzer.compute("spike_amplitudes")
analyzer.compute("amplitude_scalings")
analyzer.compute("correlograms", window_ms=50.0, bin_ms=1.0)
analyzer.compute("unit_locations", method="monopolar_triangulation")
analyzer.compute("template_similarity")
Raw lines 152 to 168
169## 6. Quality Metrics
171```python
172metric_names = ["snr", "isi_violation", "presence_ratio", "amplitude_cutoff",
173 "firing_rate", "amplitude_cv", "sliding_rp_violation"]
174analyzer.compute("quality_metrics", metric_names=metric_names)
175metrics = analyzer.get_extension("quality_metrics").get_data()
176print(metrics.head())
177```

6. Quality Metrics

metric_names = ["snr", "isi_violation", "presence_ratio", "amplitude_cutoff",
                "firing_rate", "amplitude_cv", "sliding_rp_violation"]
analyzer.compute("quality_metrics", metric_names=metric_names)
metrics = analyzer.get_extension("quality_metrics").get_data()
print(metrics.head())
Raw lines 169 to 178
179### Key metrics
181| Metric (column) | Good value | Description |
182|-----------------|------------|-------------|
183| `snr` | > 5 | Signal-to-noise ratio |
184| `isi_violations_ratio` | < 0.5 (strict: < 0.01) | Refractory period violations |
185| `presence_ratio` | > 0.9 | Fraction of recording with spikes |
186| `amplitude_cutoff` | < 0.1 | Estimated missed spikes |
187| `firing_rate` | > 0.1 Hz | Average firing rate |

Key metrics

Metric (column)Good valueDescription
snr> 5Signal-to-noise ratio
isi_violations_ratio< 0.5 (strict: < 0.01)Refractory period violations
presence_ratio> 0.9Fraction of recording with spikes
amplitude_cutoff< 0.1Estimated missed spikes
firing_rate> 0.1 HzAverage firing rate
Raw lines 179 to 188
189## 7. Curation
191### Threshold-based
193```python
194query = "(amplitude_cutoff < 0.1) & (isi_violations_ratio < 0.5) & (presence_ratio > 0.9)"
195good_unit_ids = metrics.query(query).index.values
196```
198For local `allen` / legacy `ibl` / `strict` screening presets (not IBL standards) in one call, use `scripts/compute_metrics.py`.

Threshold-based

query = "(amplitude_cutoff < 0.1) & (isi_violations_ratio < 0.5) & (presence_ratio > 0.9)"
good_unit_ids = metrics.query(query).index.values

For local allen / legacy ibl / strict screening presets (not IBL standards) in one call, use scripts/compute_metrics.py.

Raw lines 191 to 199
Runs code scripts/compute_metrics.py :198
200### Model-based (UnitRefine)
202First inspect [model compatibility and required features](AUTOMATED_CURATION.md).
203The public models advertise SI 0.102.0/sklearn 1.4.2; this environment is untested
204for their predictions. Compute model features before calling the illustrative code.
206```python
207noise_labels = sc.model_based_label_units(
208 sorting_analyzer=analyzer,
209 repo_id="SpikeInterface/UnitRefine_noise_neural_classifier",
210 trust_model=True,
211 enforce_metric_params=True,
213neural = analyzer.remove_units(noise_labels[noise_labels["prediction"] == "noise"].index)
214sua_mua = sc.model_based_label_units(
215 sorting_analyzer=neural,
216 repo_id="SpikeInterface/UnitRefine_sua_mua_classifier",
217 trust_model=True,
218 enforce_metric_params=True,
220```

Model-based (UnitRefine)

First inspect model compatibility and required features. The public models advertise SI 0.102.0/sklearn 1.4.2; this environment is untested for their predictions. Compute model features before calling the illustrative code.

noise_labels = sc.model_based_label_units(
    sorting_analyzer=analyzer,
    repo_id="SpikeInterface/UnitRefine_noise_neural_classifier",
    trust_model=True,
    enforce_metric_params=True,
)
neural = analyzer.remove_units(noise_labels[noise_labels["prediction"] == "noise"].index)
sua_mua = sc.model_based_label_units(
    sorting_analyzer=neural,
    repo_id="SpikeInterface/UnitRefine_sua_mua_classifier",
    trust_model=True,
    enforce_metric_params=True,
)
Raw lines 200 to 221
222### AI-assisted (uncertain units)
224Read API keys from the environment — never hardcode them (see [AI_CURATION.md](AI_CURATION.md)):
226```python
227import os
228from anthropic import Anthropic
230client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
231uncertain = metrics.query("snr > 3 and snr < 8").index.tolist()
232# Render each uncertain unit's summary image and ask the model to classify it.
233```

AI-assisted (uncertain units)

Read API keys from the environment — never hardcode them (see AI_CURATION.md):

import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
uncertain = metrics.query("snr > 3 and snr < 8").index.tolist()
# Render each uncertain unit's summary image and ask the model to classify it.
Raw lines 222 to 234
235## 8. Export Results

8. Export Results

Raw lines 235 to 236
237### Export to Phy
239```python
240analyzer_clean = analyzer.select_units(good_unit_ids, folder="analyzer_clean/", format="binary_folder")
241si.export_to_phy(analyzer_clean, output_folder="phy_export/",
242 compute_pc_features=True, compute_amplitudes=True)
243```

Export to Phy

analyzer_clean = analyzer.select_units(good_unit_ids, folder="analyzer_clean/", format="binary_folder")
si.export_to_phy(analyzer_clean, output_folder="phy_export/",
                 compute_pc_features=True, compute_amplitudes=True)
Raw lines 237 to 244
245### Export to NWB
247Use NeuroConv as described in [ANALYSIS.md](ANALYSIS.md); SI 0.105.0 has no
248`export_to_nwb` function.

Export to NWB

Use NeuroConv as described in ANALYSIS.md; SI 0.105.0 has no export_to_nwb function.

Raw lines 245 to 249
250### Save quality summary
252```python
253metrics.to_csv("quality_metrics.csv")
255import json
256labels = {str(uid): ("good" if uid in good_unit_ids else "other") for uid in metrics.index}
257with open("curation_labels.json", "w") as f:
258 json.dump(labels, f, indent=2)
260si.export_report(analyzer_clean, "report/", format="png")
261```

Save quality summary

metrics.to_csv("quality_metrics.csv")

import json
labels = {str(uid): ("good" if uid in good_unit_ids else "other") for uid in metrics.index}
with open("curation_labels.json", "w") as f:
    json.dump(labels, f, indent=2)

si.export_report(analyzer_clean, "report/", format="png")
Raw lines 250 to 262
263## Full Pipeline Example
265```python
266import spikeinterface.full as si
268si.set_global_job_kwargs(n_jobs=-1, chunk_duration="1s", progress_bar=True)
270# Load
271recording = si.read_spikeglx("/data/experiment/", stream_name="imec0.ap")
273# Preprocess
274rec = si.highpass_filter(recording, freq_min=400.0)
275bad_channel_ids, _ = si.detect_bad_channels(rec)
276rec = rec.remove_channels(bad_channel_ids)
277rec = si.phase_shift(rec)
278rec = si.common_reference(rec, operator="median", reference="global")
280# Motion correction
281rec = si.correct_motion(rec, preset="nonrigid_fast_and_accurate", folder="motion/")
283# Sort
284sorting = si.run_sorter("kilosort4", rec, folder="ks4/", do_correction=False)
286# Postprocess + metrics
287analyzer = si.create_sorting_analyzer(sorting, rec, sparse=True, format="binary_folder", folder="analyzer/")
288analyzer.compute(["random_spikes", "waveforms", "templates", "noise_levels",
289 "spike_amplitudes", "correlograms", "unit_locations"])
290analyzer.compute("quality_metrics",
291 metric_names=["snr", "isi_violation", "presence_ratio", "amplitude_cutoff", "firing_rate"])
292metrics = analyzer.get_extension("quality_metrics").get_data()
294# Curate
295query = "(amplitude_cutoff < 0.1) & (isi_violations_ratio < 0.5) & (presence_ratio > 0.9)"
296good_unit_ids = metrics.query(query).index.values
297print(f"Good units: {len(good_unit_ids)}/{len(metrics)}")
298```
300Or run it as a script:
302```bash
303python scripts/neuropixels_pipeline.py /data/experiment/ output/ --sorter kilosort4 --curation allen
304```

Full Pipeline Example

import spikeinterface.full as si

si.set_global_job_kwargs(n_jobs=-1, chunk_duration="1s", progress_bar=True)

# Load
recording = si.read_spikeglx("/data/experiment/", stream_name="imec0.ap")

# Preprocess
rec = si.highpass_filter(recording, freq_min=400.0)
bad_channel_ids, _ = si.detect_bad_channels(rec)
rec = rec.remove_channels(bad_channel_ids)
rec = si.phase_shift(rec)
rec = si.common_reference(rec, operator="median", reference="global")

# Motion correction
rec = si.correct_motion(rec, preset="nonrigid_fast_and_accurate", folder="motion/")

# Sort
sorting = si.run_sorter("kilosort4", rec, folder="ks4/", do_correction=False)

# Postprocess + metrics
analyzer = si.create_sorting_analyzer(sorting, rec, sparse=True, format="binary_folder", folder="analyzer/")
analyzer.compute(["random_spikes", "waveforms", "templates", "noise_levels",
                  "spike_amplitudes", "correlograms", "unit_locations"])
analyzer.compute("quality_metrics",
                 metric_names=["snr", "isi_violation", "presence_ratio", "amplitude_cutoff", "firing_rate"])
metrics = analyzer.get_extension("quality_metrics").get_data()

# Curate
query = "(amplitude_cutoff < 0.1) & (isi_violations_ratio < 0.5) & (presence_ratio > 0.9)"
good_unit_ids = metrics.query(query).index.values
print(f"Good units: {len(good_unit_ids)}/{len(metrics)}")

Or run it as a script:

python scripts/neuropixels_pipeline.py /data/experiment/ output/ --sorter kilosort4 --curation allen
Raw lines 263 to 305
Runs code python scripts/neuropixels_pipeline.py :303
306## Tips for Success
3081. **Always visualize drift** before deciding on motion correction.
3092. **Budget caching** to balance disk use and recomputation.
3103. **Compare multiple sorters** for critical experiments.
3114. **Review uncertain units manually** — don't trust automated curation blindly.
3125. **Document parameters and model repo IDs** for reproducibility.
3136. **Use a GPU** for Kilosort4.

Tips for Success

  1. Always visualize drift before deciding on motion correction.
  2. Budget caching to balance disk use and recomputation.
  3. Compare multiple sorters for critical experiments.
  4. Review uncertain units manually — don't trust automated curation blindly.
  5. Document parameters and model repo IDs for reproducibility.
  6. Use a GPU for Kilosort4.
Raw lines 306 to 313