SpikeInterface 0.97.0 release notes

10th February 2023

Main change:

  • Implemented “sparse” waveforms in core and propagated to all modules (#1079, #1136, #1152):

    • Add waveform extractor save() function and Zarr backend (#1079)

    • Dense waveforms can be saved as sparse with the save() function

    • Sparse waveforms can be extracted directly with sparse=True

  • Added IBL preprocessing tools for Neuropixels (#1025, #1059, #1070):

    • Bad channel detection

    • Channel interpolation

    • High-pass spatial filter

  • Add subfolder for sorter output (“output_folder/sorter_output”) and sorting_info field to Sorting,
    so that “sorter_output” can be deleted without deleting metadata and sorting_info (#1149, #1150)

  • Refactor Quality Metrics module (#1155, #1177, #1197):

    • Improve parameter handling

    • Unified naming and conventions

    • Add missing metrics: drift, amp medians, sliding_rp_violations

  • Extended Motion estimation and motion correction, which is now ready to use (#1129, #1108, #1183)

  • Removed deprecated functions/modules (#1261):

    • toolkit module

    • symmetrize argument from compute_correlograms

    • localize_units from postprocessing

  • Major rewrite of the docs, with extensive module-specific documentation (#1264, #1265, #1267, #1268, #1269, #1273, #1275, #1276, #1277, #1281, #1284, #1285, #1287, #1288, #1291)

core:

  • Added possibility of extracting unfiltered waveforms (#1046)

  • Prefer folder` argument and make zarr_path` deprecated in save() function (#1049)

  • Fix bug ‘int16’ is not a JSON serializable (#1069)

  • Add waveform extractor save() function and Zarr backend (#1079)

  • ChunkRecordingExecutor: add max_thread_per_process to limit the number of thread using the module threadpool_limits (#1086)

  • Correct channel selection in tests when mono-channel recording (#1096)

  • Add split_job_kwargs() function (#1110)

  • Improve handling of not dumpable objects (#1131)

  • Implement ChannelSparsity object in core (#1136)

  • Handle sparsity inside WaveformExtractor (#1152)

  • Use absolute folder in NpySnippetsFolder (#1171)

  • Add global_job_kwargs functions (#1176)

  • Propagate probe annotations to recording (#1208)

  • Serialization for recording and sorting objects (#1227)

  • Extend WaveformExtractor rec_attributes and propagate recordingless mode (#1258)

  • Improve serialization (#1227, #1272)

extractors:

  • NWBRecordingExtractor offset extraction (#1057)

  • fsspec for streaming NWB data (#1090)

  • Optimizing Phy-based extractors (#1153)

  • Add CellExplorerSortingExtractor (#1158)

  • Fix neuralynx sorting function and NeoBaseSorting when no streams (#1171)

  • Add option to load synchronized timestamps in OpenEphys (#1198)

  • Streaming IBL recordings (#1204, #1251)

  • Load phy properties from cluster_info.csv (#1210)

  • Fix the sample_shifts for spikeglx when not all channel are saved (#1232)

preprocessing:

  • Add IBL channel interpolation to preprocessing (#1025)

  • Add IBL high pass spatial filter to preprocessing (#1059)

  • Add IBL bad channel detection (#1070)

  • Add get_spatial_interpolation_kernel in preprocessing (#1074)

  • Add median/average artefact modes to remove_artifacts (#1106, #1143, #1250)

  • Fix phase-shift dtype (#1172)

  • Add correct_lsb function (#1262)

sorters:

  • Refactor sorter_output folder to keep sorting metadata (#1149)

  • Add sorting_info property to Sorting object (#1150)

  • Set n_jobs to -1 for sorters to propagate global job kwargs (#1207)

  • Added sample2volt for IBL version of pykilosort (#1211)

  • Correctly find parent folders for concatenate recording/sorting objects (#1230)

postprocessing:

  • Fix template_metrics select_units() and extend postprocessing tests (#1112)

  • Support recordingless mode for PCA (#1132)

  • Improve Phy+PCA sparsity handling (#1133)

  • More flexibility to auto merge function (#1135)

  • Fix template metrics for sparse waveforms (#1196)

  • Noise level as a waveform extractor extension (#1221)

qualitymetrics:

  • Refactor amplitude_cutoff and presence_ratio metrics (#1142)

  • Refactor QM parameter handling (#1155)

  • Implement missing metrics (amplitude median, drift metrics) (#1177)

  • Add sliding RP violations IBL metric (#1197)

  • Quality metrics and postprocessing fixes for sparsity and global job_kwargs (#1199)

  • Protect against QM returning None (#1224)

comparison:

  • Fix networkx deprecation and clean up comparison tests cache (#1189)

curation:

  • Fixed bug in remove_duplicated_spikes with ‘random’ method (#1061)

  • Added metadata copy from parent in RemoveDuplicatedSpikesSorting and AlignSortingExtractor (#1067)

  • Add option to return redundant pairs in remove_redundant_units (#1078)

  • Allow apply_sortingview_curation to load json curation files directly (#1104)

  • Improve MergeUnitsSorting to have several merge at the same time (#1244)

widgets:

  • Increase sv timeresies tile-size (#1101)

  • Updates for SV sorting summmary (#1165)

  • Promote distinctipy color as a second choice because it is slow (#1222)

sortingcomponents:

  • Refactor detect_peak() and refactor method_kwargs in components (#1105)

  • Port kilosort motion estimation (iterative_template registration) (#1108)

  • Refactor motion estimation (#1183)

  • Initial refactoring for node pipeline (#1209, #1271)

  • Fix gaussian computation in motion (#1257)

docs:

  • Fix broken spikeinterface-gui links (#1072)

  • Enhance docker docs (#1081, #1087)

  • Refactor docs (#1264, #1265, #1267, #1268, #1269, #1273, #1275, #1276, #1277, #1281, #1284, #1285, #1287, #1288, #1291)

ci:

  • Fix CI python version (#1141)

  • Fix code coverage and ignore skipped sorters (#1220)

  • Time report for full tests (#1297, #1299)