Skill Tracer
Skills › Scientific Agent Skills › neuropixels-analysis › SPIKE_SORTING.md

SPIKE_SORTING.md

112 lines · skills/neuropixels-analysis/references/SPIKE_SORTING.md at 5f2bc03

Open on GitHub
1# Spike sorting
2
3SpikeInterface 0.105.0 wrapper contracts reviewed 2026-10-01. Sorter invocations
4below are illustrative: no Kilosort, MATLAB, CUDA, container image or large real
5recording was run in this audit. Synthetic tests cover wrapper routing and
6postprocessing, not sorter accuracy or performance.
7

Spike sorting

SpikeInterface 0.105.0 wrapper contracts reviewed 2026-10-01. Sorter invocations below are illustrative: no Kilosort, MATLAB, CUDA, container image or large real recording was run in this audit. Synthetic tests cover wrapper routing and postprocessing, not sorter accuracy or performance.

Raw lines 1 to 7
8## Choose an implementation and its environment
9
10| Sorter | Requirements and scope |
11|---|---|
12| Kilosort4 | External `kilosort` and PyTorch; CUDA recommended for Neuropixels-scale work. The SI wrapper also supports `torch_device="cpu"`. |
13| Kilosort3 / 2.5 | Legacy external MATLAB or compiled/container environment, with its CUDA requirements. |
14| Spykingcircus2 / Tridesclous2 | Implemented in SpikeInterface, but require optional sorting dependencies. They are not the legacy `spykingcircus` package. |
15| Mountainsort5 | External `mountainsort5` package; CPU workflow. |
17Install only the selected sorter in an isolated environment, following its upstream
18instructions. `si.available_sorters()` lists wrappers; `si.installed_sorters()` is
19an installation check, not a successful runtime test. Kilosort4's full parameter
20list is assembled from the installed Kilosort package. Without it,
21`si.get_default_sorter_params("kilosort4")` is incomplete and emits a warning.

Choose an implementation and its environment

SorterRequirements and scope
Kilosort4External kilosort and PyTorch; CUDA recommended for Neuropixels-scale work. The SI wrapper also supports torch_device="cpu".
Kilosort3 / 2.5Legacy external MATLAB or compiled/container environment, with its CUDA requirements.
Spykingcircus2 / Tridesclous2Implemented in SpikeInterface, but require optional sorting dependencies. They are not the legacy spykingcircus package.
Mountainsort5External mountainsort5 package; CPU workflow.

Install only the selected sorter in an isolated environment, following its upstream instructions. si.available_sorters() lists wrappers; si.installed_sorters() is an installation check, not a successful runtime test. Kilosort4's full parameter list is assembled from the installed Kilosort package. Without it, si.get_default_sorter_params("kilosort4") is incomplete and emits a warning.

Raw lines 8 to 22
23## Kilosort4
25```python
26import spikeinterface.full as si
27# Requires a working Kilosort4 installation and a validated AP recording.
28params = si.get_default_sorter_params("kilosort4")
29print(params)
30sorting = si.run_sorter("kilosort4", rec, folder="ks4_output/",
31 do_CAR=False, # Already referenced externally.
32 do_correction=True, # Only if input was not motion-corrected.
33 verbose=True)
34```
36The wrapper obtains sample rate and channel count from the recording. `batch_size`
37is in samples; `nblocks` configures internal registration; `Th_universal` and
38`Th_learned` are detection thresholds. Do not copy obsolete `spkTh`, `nfilt_factor`
39or `save_extra_kwargs` into KS4. Its wrapper calls the last option `save_extra_vars`.
40Do not set `skip_kilosort_preprocessing=True` solely because filtering was done:
41that also affects whitening and must match the sorter's input assumptions.
42The current wrapper can use a binary file or an extractor (`use_binary_file`);
43a manual binary cache is a performance/storage choice.
45For externally corrected input, set `do_correction=False` (also for KS2.5/KS3).
46Kilosort3 uses `car=False`, not the KS4 `do_CAR` spelling. Inspect the wrapper
47parameters after any version change. Do not infer duplicate neuron identity from
48unit counts or fix apparent over-splitting merely by increasing template spacing.

Kilosort4

import spikeinterface.full as si
# Requires a working Kilosort4 installation and a validated AP recording.
params = si.get_default_sorter_params("kilosort4")
print(params)
sorting = si.run_sorter("kilosort4", rec, folder="ks4_output/",
                        do_CAR=False,  # Already referenced externally.
                        do_correction=True,  # Only if input was not motion-corrected.
                        verbose=True)

The wrapper obtains sample rate and channel count from the recording. batch_size is in samples; nblocks configures internal registration; Th_universal and Th_learned are detection thresholds. Do not copy obsolete spkTh, nfilt_factor or save_extra_kwargs into KS4. Its wrapper calls the last option save_extra_vars. Do not set skip_kilosort_preprocessing=True solely because filtering was done: that also affects whitening and must match the sorter's input assumptions. The current wrapper can use a binary file or an extractor (use_binary_file); a manual binary cache is a performance/storage choice.

For externally corrected input, set do_correction=False (also for KS2.5/KS3). Kilosort3 uses car=False, not the KS4 do_CAR spelling. Inspect the wrapper parameters after any version change. Do not infer duplicate neuron identity from unit counts or fix apparent over-splitting merely by increasing template spacing.

Raw lines 23 to 49
50## CPU options
52```python
53# Referenced and filtered input; retain the sorter's whitening.
54sorting_sc2 = si.run_sorter("spykingcircus2", rec_corrected, folder="sc2/",
55 apply_preprocessing=False, apply_motion_correction=False)
56sorting_ms5 = si.run_sorter("mountainsort5", rec_corrected, folder="ms5/",
57 filter=False, whiten=True, scheme="2")
58```
60Spykingcircus2 detection and selection parameters are nested dictionaries in
610.105.0, not top-level `detect_threshold`/`selection_method`. Get the current
62schema first and change only the relevant nested entries. Filtering, referencing,
63whitening and motion correction are separate stages; disable only those already
64performed. The bundled preprocessing script does **not** whiten.

CPU options

# Referenced and filtered input; retain the sorter's whitening.
sorting_sc2 = si.run_sorter("spykingcircus2", rec_corrected, folder="sc2/",
                            apply_preprocessing=False, apply_motion_correction=False)
sorting_ms5 = si.run_sorter("mountainsort5", rec_corrected, folder="ms5/",
                            filter=False, whiten=True, scheme="2")

Spykingcircus2 detection and selection parameters are nested dictionaries in 0.105.0, not top-level detect_threshold/selection_method. Get the current schema first and change only the relevant nested entries. Filtering, referencing, whitening and motion correction are separate stages; disable only those already performed. The bundled preprocessing script does not whiten.

Raw lines 50 to 65
66## Comparisons, segments, and unit identity
68```python
69comparison = si.compare_multiple_sorters([sorting_a, sorting_b], name_list=["A", "B"])
70consensus = comparison.get_agreement_sorting(minimum_agreement_count=2)
71```
73Agreement does not prove correctness; sorters can share errors. There is no
74`si.create_ensemble_sorting` convenience API in this release. Compare matched
75spike trains and waveforms, and check the units missed by consensus.
77Kilosort4's wrapper requires one segment. To concatenate recordings, first verify
78identical sample rate, contact mapping and preprocessing; preserve each original
79boundary and time origin. Use `si.concatenate_recordings(recordings)` and
80`si.split_sorting(sorting, recording_concat)` when splitting the result back.
81Discontinuous sessions need an explicit plan for gaps/artifacts. Independently
82sorted chunks do not preserve unit IDs across chunks. For a single recording,
83use `recording.frame_slice(start_frame, end_frame)`, not `split_by_times`.

Comparisons, segments, and unit identity

comparison = si.compare_multiple_sorters([sorting_a, sorting_b], name_list=["A", "B"])
consensus = comparison.get_agreement_sorting(minimum_agreement_count=2)

Agreement does not prove correctness; sorters can share errors. There is no si.create_ensemble_sorting convenience API in this release. Compare matched spike trains and waveforms, and check the units missed by consensus.

Kilosort4's wrapper requires one segment. To concatenate recordings, first verify identical sample rate, contact mapping and preprocessing; preserve each original boundary and time origin. Use si.concatenate_recordings(recordings) and si.split_sorting(sorting, recording_concat) when splitting the result back. Discontinuous sessions need an explicit plan for gaps/artifacts. Independently sorted chunks do not preserve unit IDs across chunks. For a single recording, use recording.frame_slice(start_frame, end_frame), not split_by_times.

Raw lines 66 to 84
85## Save, review, and reload
87```python
88sorting.save(folder="sorting/")
89sorting = si.load("sorting/")
90# Or read the wrapper's original output:
91sorting = si.read_sorter_folder("ks4_output/")
92```
94Compute a SortingAnalyzer as described in [ANALYSIS.md](ANALYSIS.md), then export
95all units for manual Phy review. `si.read_phy("phy_export/")` returns the curated
96sorting including splits/merges; inspect `get_property_keys()` and cluster-group
97properties rather than assuming `quality`. Recompute analyzer extensions after
98manual changes. There is no `si.apply_phy_curation` helper in this release.
100Container invocation uses `docker_image=True` or an explicitly pinned supported
101image, or `singularity_image=...`; it still needs the host runtime/GPU setup.
102A mutable `latest` tag is not a reproducible environment.
104Sources: [SI sorter API](https://spikeinterface.readthedocs.io/en/stable/api.html),
105[KS4 wrapper](https://github.com/SpikeInterface/spikeinterface/blob/0.105.0/src/spikeinterface/sorters/external/kilosort4.py),
106[Kilosort installation](https://kilosort.readthedocs.io/en/latest/README.html),
107[Kilosort parameters](https://github.com/MouseLand/Kilosort/blob/main/kilosort/parameters.py).
109SpikeInterface 0.105.0 has an observed `read_phy` bug for exported nonnumeric
110unit IDs (`np.isnan` TypeError). Phy export preserves `cluster_si_unit_ids.tsv`;
111keep that mapping and use a validated importer/fixed release for string-ID
112readback. Numeric-ID Phy export/reload was tested on synthetic data.

Save, review, and reload

sorting.save(folder="sorting/")
sorting = si.load("sorting/")
# Or read the wrapper's original output:
sorting = si.read_sorter_folder("ks4_output/")

Compute a SortingAnalyzer as described in ANALYSIS.md, then export all units for manual Phy review. si.read_phy("phy_export/") returns the curated sorting including splits/merges; inspect get_property_keys() and cluster-group properties rather than assuming quality. Recompute analyzer extensions after manual changes. There is no si.apply_phy_curation helper in this release.

Container invocation uses docker_image=True or an explicitly pinned supported image, or singularity_image=...; it still needs the host runtime/GPU setup. A mutable latest tag is not a reproducible environment.

Sources: SI sorter API, KS4 wrapper, Kilosort installation, Kilosort parameters.

SpikeInterface 0.105.0 has an observed read_phy bug for exported nonnumeric unit IDs (np.isnan TypeError). Phy export preserves cluster_si_unit_ids.tsv; keep that mapping and use a validated importer/fixed release for string-ID readback. Numeric-ID Phy export/reload was tested on synthetic data.

Raw lines 85 to 112