Run spike sorting by property

Sometimes you may want to spike sort different electrodes separately. For example your probe can have several channel groups (for example tetrodes) or you might want to spike sort different brain regions separately, In these cases, you can spike sort by property.

import numpy as np
import spikeinterface.extractors as se
import spikeinterface.sorters as ss

Sometimes, you might want to sort your data depending on a specific property of your recording channels.

For example, when using multiple tetrodes, a good idea is to sort each tetrode separately. In this case, channels belonging to the same tetrode will be in the same ‘group’. Alternatively, for long silicon probes, such as Neuropixels, you could sort different areas separately, for example hippocampus and thalamus.

All this can be done by sorting by ‘property’. Properties can be loaded to the recording channels either manually (using the set_channel_property method), or by using a probe file. In this example we will create a 16 channel recording and split it in four channel groups (tetrodes).

Let’s create a toy example with 16 channels (the dumpable=True dumps the extractors to a file, which is required for parallel sorting):

recording, sorting_true = se.toy_example(duration=[10.], num_segments=1, num_channels=16)
# make dumpable
recording= recording.save()

Out:

Use cache_folder=/tmp/spikeinterface_cache/tmp8glh4q8j/2DGJBA0J
write_binary_recording with n_jobs 1  chunk_size None

Initially all channel are in the same group.

print(recording.get_channel_groups())

Out:

[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

Lets now change the probe mapping and assign a 4 tetrodes to this recording. for this we will use the probeinterface module and create a ProbeGroup containing for dummy tetrode.

from probeinterface import generate_tetrode, ProbeGroup

probegroup = ProbeGroup()
for i in range(4):
    tetrode = generate_tetrode()
    tetrode.set_device_channel_indices(np.arange(4) + i * 4)
    probegroup.add_probe(tetrode)

now our new recording contain 4 groups

recording_4_tetrodes = recording.set_probegroup(probegroup, group_mode='by_probe')

# get group
print(recording_4_tetrodes.get_channel_groups())
# similar to this
print(recording_4_tetrodes.get_property('group'))

Out:

[0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3]
[0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3]

this “group” property can be use to split our new recording into 4 recording we get a list of 4 ChannelSliceRecording (ex “sub-recording”) This is done without any copy, each ChannelSliceRecording is a view of the parent recording

Note that here we use ‘group’ for splitting but it could be done on any property.

recordings = recording_4_tetrodes.split_by(property='group')
print(recordings)

Out:

{0: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s, 1: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s, 2: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s, 3: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s}

We can also get a dict instead of the list which is easier to handle group keys.

recordings = recording_4_tetrodes.split_by(property='group', outputs='dict')
print(recordings)

Out:

{0: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s, 1: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s, 2: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s, 3: ChannelSliceRecording: 4 channels - 1 segments - 30.0kHz - 10.000s}

We can now use the run_sorters() function instead of the run_sorter(). This function can run several sorters on several recording with different parallel engines.

here we use engine ‘loop’ but we could use also ‘joblib’ or ‘dask’ for multi process or multi node computing. have a look to the documentation of this function that handle many cases.

sorter_list = ['tridesclous']
working_folder = 'sorter_outputs'
results = ss.run_sorters(sorter_list, recordings, working_folder,
            engine='loop', with_output=True, mode_if_folder_exists='overwrite')

Out:

/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1582: UserWarning: Trying to register the cmap 'rocket' which already exists.
  mpl_cm.register_cmap(_name, _cmap)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1583: UserWarning: Trying to register the cmap 'rocket_r' which already exists.
  mpl_cm.register_cmap(_name + "_r", _cmap_r)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1582: UserWarning: Trying to register the cmap 'mako' which already exists.
  mpl_cm.register_cmap(_name, _cmap)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1583: UserWarning: Trying to register the cmap 'mako_r' which already exists.
  mpl_cm.register_cmap(_name + "_r", _cmap_r)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1582: UserWarning: Trying to register the cmap 'icefire' which already exists.
  mpl_cm.register_cmap(_name, _cmap)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1583: UserWarning: Trying to register the cmap 'icefire_r' which already exists.
  mpl_cm.register_cmap(_name + "_r", _cmap_r)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1582: UserWarning: Trying to register the cmap 'vlag' which already exists.
  mpl_cm.register_cmap(_name, _cmap)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1583: UserWarning: Trying to register the cmap 'vlag_r' which already exists.
  mpl_cm.register_cmap(_name + "_r", _cmap_r)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1582: UserWarning: Trying to register the cmap 'flare' which already exists.
  mpl_cm.register_cmap(_name, _cmap)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1583: UserWarning: Trying to register the cmap 'flare_r' which already exists.
  mpl_cm.register_cmap(_name + "_r", _cmap_r)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1582: UserWarning: Trying to register the cmap 'crest' which already exists.
  mpl_cm.register_cmap(_name, _cmap)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/seaborn/cm.py:1583: UserWarning: Trying to register the cmap 'crest_r' which already exists.
  mpl_cm.register_cmap(_name + "_r", _cmap_r)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/tridesclous/dip.py:61: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  (np.less(d, unif_dips).sum() + 1) / (np.float(numt) + 1)
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:275: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  alg = KDTreeBoruvkaAlgorithm(tree, min_samples, metric=metric,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:56: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  condensed_tree = condense_tree(single_linkage_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/hdbscan/hdbscan_.py:59: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  labels, probabilities, stabilities = get_clusters(condensed_tree,
/home/docs/checkouts/readthedocs.org/user_builds/spikeinterface/conda/latest/lib/python3.8/site-packages/tridesclous/dip.py:61: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  (np.less(d, unif_dips).sum() + 1) / (np.float(numt) + 1)

the output is a dict with all combinations of (group, sorter_name)

from pprint import pprint
pprint(results)

Out:

{('0', 'tridesclous'): TridesclousSortingExtractor: 2 units - 1 segments - 30.0kHz,
 ('1', 'tridesclous'): TridesclousSortingExtractor: 2 units - 1 segments - 30.0kHz,
 ('2', 'tridesclous'): TridesclousSortingExtractor: 1 units - 1 segments - 30.0kHz,
 ('3', 'tridesclous'): TridesclousSortingExtractor: 1 units - 1 segments - 30.0kHz}

Total running time of the script: ( 0 minutes 3.754 seconds)

Gallery generated by Sphinx-Gallery