YASS: Yet Another Spike Sorter
Spike sorting is a critical first step in extracting neural signals from large-scale electrophysiological data. This manuscript describes an efficient, reliable pipeline for spike sorting on dense multi-electrode arrays (MEAs), where neural signals appear across many electrodes and spike sorting currently represents a major computational bottleneck. We present several new techniques that make dense MEA spike sorting more robust and scalable. Our pipeline is based on an efficient multistage "triage-then-cluster-then-pursuit" approach that initially extracts only clean, high-quality waveforms from the electrophysiological time series by temporarily skipping noisy or "collided" events (representing two neurons firing synchronously). This is accomplished by developing a neural network detection method followed by efficient outlier triaging. The clean waveforms are then used to infer the set of neural spike waveform templates through nonparametric Bayesian clustering. Our clustering approach adapts a "coreset" approach for data reduction and uses efficient inference methods in a Dirichlet process mixture model framework to dramatically improve the scalability and reliability of the entire pipeline. The "triaged" waveforms are then finally recovered with matching-pursuit deconvolution techniques. The proposed methods improve on the state-of-the-art in terms of accuracy and stability on both real and biophysically-realistic simulated MEA data. Furthermore, the proposed pipeline is efficient, learning templates and clustering faster than real-time for a ' 500-electrode dataset, largely on a single CPU core.
[UPDATE AFTER AUTHOR RESPONSE]
The authors' response confirms my rating – it's a valuable paper. I'm optimistic that they will address mine and the other reviewers' concerns in their revision.
[ORIGINAL REVIEW]
The paper describes YASS: Yet Another Spike Sorter, a well-engineered pipeline for sorting multi-electrode array (MEA) recordings. The approach is sound, combining many state-of-the-art approaches into a complete pipeline. The paper is well written, follows a clear structure and provides convincing evidence that the work indeed advances the state of the art for sorting retinal MEA data. 
While the paper is already a valuable contribution as is, I think it has much greater potential if the authors address some of the issues detailed below. In brief, my main concerns are
(1) the lack of publicly available code,
(2) the poor description of the neural network detection method,
(3) issues with applying the pipeline to cortical data.
Details:
(1) Code is not available (or at least the manuscript does not provide a URL). Since spike sorting is at this point mainly an engineering problem (but a non-trivial one), a mere description of the approach is only half as valuable as the actual code implementing the pipeline. Thus, I strongly encourage the authors to go the extra mile and make the code available.
(2) Neural network spike detection. This part seems to be the only truly innovative one (all other components of the pipeline have been described/used before). However, it remains unclear to me how the authors generated their training data. Section C.2 describes different ways of generating training data, but it is not clear which one (or which combination) the authors use.
(a) Using pre-existing sorts. 
First, most labs do *not* have existing, properly sorted data available when moving to dense MEAs, because they do not have a pipeline for sorting such array data and – as the authors point out – existing methods do not scale properly. 
Second, it is not clear to me how existing sorts should help training a more robust neural network for detection. Did the authors inspect every single waveform snippet and labeled it as clean or not? If not, based on what algorithm did they decide which waveform snippets in the training data are clean? Why do they need to train a neural network instead of just using this algorithm? How do they deal with misalignments, which create label noise?
If using pre-existing sorts is what the authors did, they need to provide more information on how exactly they did it and why it works. In the current form, their work cannot be reproduced.
(b) Generating synthetic training data by superimposing waveform templates on background noise. This could be a reasonable approach. Is it used for data augmentation or not at all and just described as a potential alternative? What is the evidence that this approach is useful? The synthetic data may not be representative of real recordings.
(3) Generalization to cortical data. I am quite confident that the pipeline works well for retinal data, but I doubt that it will do so for cortical data (some arguments below). I think this limitation needs to be discussed and acknowledged more explicitly (abstract, intro, conclusions).
(a) In cortical recordings, waveform drift is a serious issue that arises in pretty much all non-chronic recordings (and chronically working high-density MEAs are still to be demonstrated). Thus, modeling drift is absolutely crucial for recordings that last longer than a few minutes.
(b) Getting good training data for the NN detection is more difficult. Good ground truth (or well validated data such as described in appendix I) is not available and generating synthetic data as described in C.2 is not necessarily realistic, since background noise is caused by spikes as well and neurons often fire in highly correlated manners (thus rendering the approach of overlaying templates on spike-free noise problematic).
Minor comments:
- Fig. 3 bottom panel: Y axis is strange. Does 1^-x mean 10^-x? Also, it exaggerates tiny accuracy differences between 0.99 and 0.999, where both methods are essentially perfect.
- The authors use spatially whitened data (according to section 2.1), but I did not find a description of the spatial whitening procedure in the manuscript or supplement.