Published as a conference paper at ICLR 2018 SYNTHESIZING REALISTIC NEURAL POPULATION ACTIVITY PATTERNS USING GENERATIVE ADVERSARIAL NETWORKS
The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing. Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons. We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from parameter sharing in the temporal domain. We demonstrate that our proposed GAN, which we termed Spike-GAN, generates spike trains that match accurately the first-and second-order statistics of datasets of tens of neurons and also approximates well their higher-order statistics. We applied Spike-GAN to a real dataset recorded from salamander retina and showed that it performs as well as state-ofthe-art approaches based on the maximum entropy and the dichotomized Gaussian frameworks. Importantly, Spike-GAN does not require to specify a priori the statistics to be matched by the model, and so constitutes a more flexible method than these alternative approaches. Finally, we show how to exploit a trained Spike-GAN to construct 'importance maps' to detect the most relevant statistical structures present in a spike train. Spike-GAN provides a powerful, easy-to-use technique for generating realistic spiking neural activity and for describing the most relevant features of the large-scale neural population recordings studied in modern systems neuroscience.
[Summary of paper] The paper presents a method for simulating spike trains from populations of neurons which match empirically measured multi-neuron recordings. They set up a Wasserstein-GAN and train it on both synthetic and real multi-neuron recordings, using data from the salamander retina. They find that their method (Spike-GAN) can produce spike trains that visually look like the original data, and which have low-order statistics (firing rates, correlations, time-lagged-correlations, total sum of population activity) which matches those of the original data. They emphasize that their network architecture is 'semi-convolutional', i.e. convolutional in time but not across neurons. Finally, they suggest a way to analyse the fitted networks in order to gain insights into what the 'relevant' neural features are, and illustrate it on synthetic data into which they embedded these features.
[Originality] This paper falls into the category of papers that do a next obvious thing ("GANs have not been applied to population spike trains yet"), and which do it pretty well: If one wants to create simulated neural activity data which matches experimentally observed one, then this method indeed seems to do that. As far as I know, this would be the first peer-reviewed application of GANs to multi-neuron recordings of neural data (but see https://arxiv.org/abs/1707.04582 for an arxiv paper, not cited here-- should be discussed at least).  On a technical level, there is very little to no innovation here -- while the authors emphasise their 'semi-convolutional' network architecture, this is obviously the right architecture to use for multivariate time-series data, and in itself not a big technical novel. Therefore, the paper should really be evaluate as an `application' paper, and be assessed in terms of i) how important the application is, ii) how clearly it is presented, and iii) how convincing the results are relative to state of the art. 
i) [Importance of problem, potential significance] Finding statistical models for modelling and simulating population spike trains is a topic which is extensively studied in computational neuroscience, predominantly using  model-based approaches using MaxEnt models, GLMs or latent variable models. These models are typically simple and restricted, and certainly fall short of capturing the full complexity of neural data. Thus, better, and more flexible solutions for this problem would certainly be very welcome, and have an immediate impact in this community.  However, I think that the approach based on GANs actually has two shortcomings which are not stated by the authors, and which possibly limit the impact of the method: First, statistical models of neural spike trains are often used to compute probabilities e.g. for decoding analyses— this is difficult or impossible for GANs. Second, one most often does not want to simulate data which match a specific recording, but rather which have specified statistics (e.g. firing rates and correlations)— the method here is based on fitting a particular data-set, and it is actually unclear to me when that will be useful.
ii) [Clarity] The methods are presented and explained clearly and cleanly. In my view, too much emphasis is given to highlighting the ‘semi-convolutional’ network, and, conversely, practical issues (exact architectures, cost of training) should be explained more clearly, possibly in an appendix. Similarly, the method would benefit from the authors releasing their code.
iii) [Quality, advance over previous methods] The authors discuss several methods for simulating spike trains in the introduction. In their empirical comparisons, however, they completely focus on a particular model-class (maximum entropy models, ME) which they label being the ‘state-of-the-art’. This label is misleading— ME models are but one of several approaches to modelling neural spike trains, with different models having different advantages and limitations (there is no benchmark which can be used to rank them...). In particular, the only ‘gain’ of the GAN over ME  models in the results comes from their ability of the GAN to match temporal statistics. Given that the ME models used by the authors are blind to temporal correlations, this is, of course (and as pointed out by the authors) hardly surprising. How does the GAN approach fair against alternative models which do take temporal statistics into account, e.g. GLMs, or simple moment-based method e.g. Krumin et al 2009, Lyamzin 2010, Gutnisky et al 2010— setting these up would be simple, and it would provide a non-trivial baseline for the ability of spike-GAN to outperform at least these models? While it true that GANs are much more expressive than the model-based approaches used in neuroscience, a clear demonstration would have been useful.
Minor comments: 
  - p.3: The abbreviation “1D-DCGAN” is never spelled out.
  - p.3: The architecture of Spike-GAN is never explicitely given.
  - p.3: (Sec. 2.2) Statistic 2) “average time course across activity patterns” is unclear to me -- how does one select the activity patterns over which to average? Moreover, later figures do not seem to use this statistic.
  - p.4: “introduced correlations between randomly selected pairs” -- How many such pairs were formed?
  - p.7 (just above Discussion) At the beginning of this section, and for Figs. 4A,B, the texts suggests that packets fire spontaneously with a given probability. For Figs. 4C-E, a particular packet responds to a particular input. Is then the neuron population used in these figures different from the one in Figs. 4A,B? How did the authors ensure that a particular set of neurons respond to their stimulus as a packet? What stimulus did they use?
  - p.8 (Fig. 4E) Are the eight neurons with higher importance those corresponding to the packet? This is insinuated but not stated.
  - p.12 (Appendix A) 
    + The authors do not mention how they produced their “ground truth” data. (What was its firing rate? Did it include correlations? A refractory period?)
    + Generating samples from the trained Spike-GAN is ostensibly cheap. Hence it is unclear why the authors did not  produce a large enough number of samples in order to obtain a 'numerical probability', just as they did for the ground truth data? 
    + Fig. S1B: The figure shows that every sample has the same empirical frequency. This indicates more a lack of statistical power rather than any correspondence between the theoretical and empirical probabilities. This undermines the argument in the second paragraph of p.12. In the other hand, if the authors did approximate numerical probabilities for the Spike-GAN, this argument would no longer be required.
  - p.13 Fig. S1A,B: the abscissas mention “frequency”, while the ordinates mention “probability”
  - p.25 Fig. S4: This figure suggests that the first layer of the Spike-GAN critic sometimes recognizes the packet patterns in the data. However, to know whether this is true, we would need to compare this to a representation of the neurons reordered in the same way and identified by packet. I.e. one expects something something like figure like Fig. 4A, with the packets lining up with the recovered filters when neurons are ordered the same way.