Nearest Neighbor Metrics (nn_hit_rate
, nn_miss_rate
, nn_isolation
, nn_noise_overlap
)¶
Calculation¶
There are several implementations of nearest neighbor metrics which can be used to evaluate unit quality.
When calling the compute_quality_metrics()
function, the following options are available to calculate NN metrics:
The
nearest_neighbor
option will returnnn_hit_rate
andnn_miss_rate
(based on [Siegle] inspired by [Chung]).The
nn_isolation
option will return the nearest neighbor isolation metric (adapted from Chung).The
nn_noise_overlap
option will return the nearest neighbor isolation metric (adapted from Chung).
All options involve non-parametric calculations in PCA space.
nearest_neighbor
¶
The membership function, \(\rho\) is defined such that for any spike \(g_i\) in some cluster \(G\), \(\rho(g_i) = G\). Additionally, the nearest neighbour function \(n_k(g_i)\) is defined such that the output of the function is the set of \(k\) spikes which are closest to \(g_i\).
For a unit associated with cluster \(C\), a subset of spikes are randomly drawn to form the cluster \(A\). A subset of spikes which are not in \(C\) are drawn to form the cluster \(B\). Note that \(|A| = |B|\). The NN-hit rate for \(C\) is then:
Similarly, the NN-miss rate for \(C\) is:
NN-hit rate gives an estimate of contamination (an uncontaminated unit should have a high NN-hit rate). NN-miss rate gives an estimate of completeness. A more complete unit should have a low NN-miss rate.
nn_isolation
¶
The overall logic of this approach is to choose a cluster for which the isolation is to be computed, and compute the pairwise isolation score between the chosen cluster and every other cluster. The isolation score is then the minimum of the pairwise scores (the worst case).
Let A and B be two clusters from sorting.
We set \(|A| = |B|\) by subsampling as appropriate to match the size of the smaller cluster (or the max_spikes_for_nn
parameter value, if using).
We also restrict the waveforms to channels with significant signal.
The pairwise isolation between clusters A and B is then:
Note that nn_isolation is affected by the size of the clusters, so setting the max_spikes_for_nn
may aid downstream comparison of scores.
nn_noise_overlap
¶
A noise cluster is generated by randomly sampling voltage snippets from the recording. Following a similar procedure to that of the nn_isolation method, compute isolation between the cluster of interest and the generated noise cluster. noise overlap is then \(1 - NN_{\textrm{isolation}}\).
This metric gives an indication of the contamination present in the unit cluster.
Example code¶
Links to source code¶
From SpikeInterface (nearest neighbors metrics).
From SpikeInterface (nearest neighbors isolation).
From SpikeInterface (nearest neighbors noise overlap).
References¶
Cluster quality metrics computed from principal components.
- spikeinterface.qualitymetrics.pca_metrics.nearest_neighbors_metrics(all_pcs, all_labels, this_unit_id, max_spikes_for_nn, n_neighbors)¶
Calculates unit contamination based on NearestNeighbors search in PCA space.
- Parameters
- all_pcs2d array
The PCs for all spikes, organized as [num_spikes, PCs].
- all_labels1d array
The cluster labels for all spikes. Must have length of number of spikes.
- this_unit_idint
The ID for the unit to calculate these metrics for.
- max_spikes_for_nnint
The number of spikes to use, per cluster. Note that the calculation can be very slow when this number is >20000.
- n_neighborsint
The number of neighbors to use.
- Returns
- hit_ratefloat
Fraction of neighbors for target cluster that are also in target cluster.
- miss_ratefloat
Fraction of neighbors outside target cluster that are in target cluster.
Notes
A is a (hopefully) representative subset of cluster X
NN_hit(X) = 1/k sum_i=1^k |{{x in A such that ith closest neighbor is in X}}| / |A|
- spikeinterface.qualitymetrics.pca_metrics.nearest_neighbors_isolation(waveform_extractor: WaveformExtractor, this_unit_id: int, max_spikes_for_nn: int = 1000, min_spikes_for_nn: int = 10, n_neighbors: int = 5, n_components: int = 10, radius_um: float = 100, seed: int = 0)¶
Calculates unit isolation based on NearestNeighbors search in PCA space.
- Parameters
- waveform_extractorWaveformExtractor
The waveform extractor object.
- this_unit_idint
The ID for the unit to calculate these metrics for.
- max_spikes_for_nnint, optional, default: 1000
Max number of spikes to use per unit.
- min_spikes_for_nnint, optional, defalt: 10
Min number of spikes a unit must have to go through with metric computation. Units with spikes < min_spikes_for_nn gets numpy.NaN as the quality metric.
- n_neighborsint, optional, default: 5
Number of neighbors to check membership of.
- n_componentsint, optional, default: 10
The number of PC components to use to project the snippets to.
- radius_umfloat, optional, default: 100
The radius, in um, that channels need to be within the peak channel to be included.
- seedint, optional, default: 0
Seed for random subsampling of spikes.
- Returns
- nn_isolationfloat
The calculation nearest neighbor isolation metric for this_unit_id. If the unit has fewer than min_spikes_for_nn, returns numpy.NaN instead.
Notes
The overall logic of this approach is: 1) Choose a cluster 2) Compute the isolation score with every other cluster 3) Isolation score is defined as the min of (2) (i.e. ‘worst-case measure’)
The implementation of this approach is:
Let A and B be two clusters from sorting.
This is because the metric is affected by the size of the clusters being compared independently of how well-isolated they are.
We also restrict the waveforms to channels with significant signal.
See docstring for _compute_isolation for the definition of isolation score.
- spikeinterface.qualitymetrics.pca_metrics.nearest_neighbors_noise_overlap(waveform_extractor: WaveformExtractor, this_unit_id: int, max_spikes_for_nn: int = 1000, min_spikes_for_nn: int = 10, n_neighbors: int = 5, n_components: int = 10, radius_um: float = 100, seed: int = 0)¶
Calculates unit noise overlap based on NearestNeighbors search in PCA space.
- Parameters
- waveform_extractorWaveformExtractor
The waveform extractor object.
- this_unit_idint
The ID of the unit to calculate this metric on.
- max_spikes_for_nnint, optional, default: 1000
The max number of spikes to use per cluster.
- min_spikes_for_nnint, optional, defalt: 10
Min number of spikes a unit must have to go through with metric computation. Units with spikes < min_spikes_for_nn gets numpy.NaN as the quality metric.
- n_neighborsint, optional, default: 5
The number of neighbors to check membership.
- n_componentsint, optional, default: 10
The number of PC components to use to project the snippets to.
- radius_umfloat, optional, default: 100
The radius, in um, that channels need to be within the peak channel to be included.
- seedint, optional, default: 0
Random seed for subsampling spikes.
- Returns
- nn_noise_overlapfloat
The computed nearest neighbor noise estimate. If the unit has fewer than min_spikes_for_nn, returns numpy.NaN instead.
Notes
The general logic of this measure is: 1. Generate a noise cluster by randomly sampling voltage snippets from recording. 2. Subtract projection onto the weighted average of noise snippets
of both the target and noise clusters to correct for bias in sampling.
Compute the isolation score between the noise cluster and the target cluster.
As with nn_isolation, the clusters that are compared (target and noise clusters) have the same number of spikes.
See docstring for _compute_isolation for the definition of isolation score.