Validation Tutorial

After spike sorting, you might want to validate the goodness of the sorted units. This can be done using the toolkit.validation submodule, which computes several quality metrics of the sorted units.

import spikeinterface.extractors as se
import spikeinterface.toolkit as st

First, let’s create a toy example:

recording, sorting = se.example_datasets.toy_example(num_channels=4, duration=10, seed=0)

The toolkit.validation submodule has a set of functions that allow users to compute metrics in a compact and easy way. To compute a single metric, the user can simply run one of the quality metric functions as shown below. Each function as a variety of adjustable parameters that can be tuned by the user to match their data.

firing_rates = st.validation.compute_firing_rates(sorting, duration_in_frames=recording.get_num_frames())
isi_violations = st.validation.compute_isi_violations(sorting, duration_in_frames=recording.get_num_frames(), isi_threshold=0.0015)
snrs = st.validation.compute_snrs(recording=recording, sorting=sorting, max_spikes_per_unit_for_snr=1000)
nn_hit_rate, nn_miss_rate = st.validation.compute_nn_metrics(recording=recording, sorting=sorting, num_channels_to_compare=13)

To compute more than one metric at once, a user can use the compute_quality_metrics function and indicate which metrics they want to compute. This will return a dictionary of metrics or optionally a pandas dataframe.

metrics = st.validation.compute_quality_metrics(sorting=sorting, recording=recording,
                                                metric_names=['firing_rate', 'isi_violation', 'snr', 'nn_hit_rate', 'nn_miss_rate'],
                                                as_dataframe=True)

To compute metrics on only part of the recording, a user can specify specific epochs in the Recording and Sorting extractor using add_epoch and then compute the metrics on the SubRecording and SubSorting extractor given by get_epoch. In this example, we compute all the same metrics on the first half of the recording.

sorting.add_epoch(epoch_name="first_half", start_frame=0, end_frame=recording.get_num_frames()/2) #set
recording.add_epoch(epoch_name="first_half", start_frame=0, end_frame=recording.get_num_frames()/2)
subsorting = sorting.get_epoch("first_half")
subrecording = recording.get_epoch("first_half")
metrics_first_half = st.validation.compute_quality_metrics(sorting=subsorting, recording=subrecording,
                                                           metric_names=['firing_rate', 'isi_violation', 'snr', 'nn_hit_rate', 'nn_miss_rate'],
                                                           as_dataframe=True)

print("Metrics full recording")
print(metrics)
print('\n')
print("Metrics first half recording")
print(metrics_first_half)

Out:

Metrics full recording
    firing_rate  isi_violation        snr  nn_hit_rate  nn_miss_rate
1           2.2            0.0  14.190699     1.000000      0.000000
2           2.6            0.0   6.252029     0.974359      0.004651
3           2.2            0.0   6.107215     0.878788      0.007610
4           2.5            0.0  15.809131     1.000000      0.004630
5           2.5            0.0   3.781591     0.893333      0.018519
6           2.7            0.0   3.945409     0.938272      0.009346
7           2.2            0.0  28.892361     1.000000      0.000000
8           2.2            0.0   7.082476     0.954545      0.000000
9           2.8            0.0   8.048143     0.964286      0.003130
10          2.2            0.0   4.093077     0.893939      0.007610


Metrics first half recording
    firing_rate  isi_violation        snr  nn_hit_rate  nn_miss_rate
1           1.4            0.0  14.225538     1.000000      0.000000
2           3.2            0.0   6.267378     0.979167      0.011494
3           3.2            0.0   6.092624     0.895833      0.011494
4           2.4            0.0  15.771360     1.000000      0.000000
5           2.6            0.0   3.772556     0.820513      0.019608
6           3.6            0.0   3.932851     0.907407      0.029240
7           2.8            0.0  28.800394     1.000000      0.000000
8           2.2            0.0   7.059932     0.909091      0.000000
9           2.8            0.0   8.035229     0.976190      0.002825
10          2.2            0.0   4.086510     0.787879      0.008264