Bayesian entropy estimation for binary spike train data using parametric prior knowledge
Shannon's entropy is a basic quantity in information theory, and a fundamental building block for the analysis of neural codes. Estimating the entropy of a discrete distribution from samples is an important and difficult problem that has received considerable attention in statistics and theoretical neuroscience. However, neural responses have characteristic statistical structure that generic entropy estimators fail to exploit. For example, existing Bayesian entropy estimators make the naive assumption that all spike words are equally likely a priori, which makes for an inefficient allocation of prior probability mass in cases where spikes are sparse. Here we develop Bayesian estimators for the entropy of binary spike trains using priors designed to flexibly exploit the statistical structure of simultaneouslyrecorded spike responses. We define two prior distributions over spike words using mixtures of Dirichlet distributions centered on simple parametric models. The parametric model captures high-level statistical features of the data, such as the average spike count in a spike word, which allows the posterior over entropy to concentrate more rapidly than with standard estimators (e.g., in cases where the probability of spiking differs strongly from 0.5). Conversely, the Dirichlet distributions assign prior mass to distributions far from the parametric model, ensuring consistent estimates for arbitrary distributions. We devise a compact representation of the data and prior that allow for computationally efficient implementations of Bayesian least squares and empirical Bayes entropy estimators with large numbers of neurons. We apply these estimators to simulated and real neural data and show that they substantially outperform traditional methods.
Paper summary: The paper describes two novel 
      entropy estimators for binary neural spike words. The estimators are 
      Bayesian and make use of a mixture-of-Dirichlet prior. The distribution is 
      hierarchical with a count distribution as the base measure of the 
      Dirichlet distribution. The authors evaluate their methods on artificial 
      data and on data recorded simultaneously from retinal ganglion cells and 
      compare them to established entropy estimators. They show that their 
      estimators need less samples for accurate estimations. Finally, they apply 
      their method to quantify temporal synergy in retinal ganglion cell data. 
      Quality: The Bayesian entropy estimators are powerful, 
      elegantly evading the curse of dimensionality. By including prior 
      information about the structure of the problem, the method reduces the 
      number of required samples. On the synthetic data, it is not 
      surprising that the proposed estimators outperform the alternative 
      estimators, since the distributions of the word data follow the model 
      structure of the estimators. The performance on the data recorded 
      simultaneously from retinal ganglion cells is impressive. It is 
      not clear, though, how well the methods would do on other neural data. The 
      method contains the critical underlying assumption that the word 
      distribution is well characterized by the overall count distribution. For 
      the retinal ganglion cell data this is apparently the case, but further 
      evaluations will have to show whether or not this will also hold in 
      general. It might be worth to mention this problem with a sentence in the 
      discussion. In any case, the new estimators are certainly very useful. 
      Clarity: The paper is nicely written. 
      Originality: The proposed entropy estimators extend 
      the work by Nemenman et al., NIPS 2002 by including prior knowledge about 
      the structure of the spike trains. The general idea is similar to that of 
      the raster marginals model (Okun et al., J Neurosci 2012) in that the 
      total spike count distribution is used as a simplification to evade the 
      curse of dimensionality. Significance: Entropy 
      estimation is a very important problem, because information quantification 
      is a central problem of neural coding analyses. The demonstrated 
      performance gain compared to alternative methods is impressive. 
      Minor points: 080: In Section 3 introduce -> In Section 
      3, we introduce 163: Citation missing 232: Incomplete sentence 
      234: the estimator fast-to-compute -> the estimator is 
      fast-to-compute 290: We only then need only -> We then need only 
      295: for -> form Figures 4 and 5: DCnt -> DCt