Universal models for binary spike patterns using centered Dirichlet processes
Probabilistic models for binary spike patterns provide a powerful tool for understanding the statistical dependencies in large-scale neural recordings. Maximum entropy (or "maxent") models, which seek to explain dependencies in terms of low-order interactions between neurons, have enjoyed remarkable success in modeling such patterns, particularly for small groups of neurons. However, these models are computationally intractable for large populations, and low-order maxent models have been shown to be inadequate for some datasets. To overcome these limitations, we propose a family of "universal" models for binary spike patterns, where universality refers to the ability to model arbitrary distributions over all 2 m binary patterns. We construct universal models using a Dirichlet process centered on a well-behaved parametric base measure, which naturally combines the flexibility of a histogram and the parsimony of a parametric model. We derive computationally efficient inference methods using Bernoulli and cascaded logistic base measures, which scale tractably to large populations. We also establish a condition for equivalence between the cascaded logistic and the 2nd-order maxent or "Ising" model, making cascaded logistic a reasonable choice for base measure in a universal model. We illustrate the performance of these models using neural data.
Review for #1163 “Universal models for binary spike 
      patterns using centered Dirichlet processes.” The goal of this 
      paper is to provide a more accurate method for modeling the distribution 
      of binary spike patterns over a population of neurons. Essentially what 
      the authors are trying to do is improve upon parametric models of the 
      pattern distribution (such as a Bernoulli, cascaded logistic or Ising 
      model) by allowing for deviations from the parametric model (or base 
      model) if they are justified by the data. The involves postulating a 
      Dirichlet process centered upon the base model and fitting the parameters 
      of the base model and the concentration parameter of the Dirichlet process 
      via gradient ascent (although I imagine other methods could be used for 
      fitting). Intuitively this constitutes fitting a type of weighted average 
      between the probability distribution of the base model and the pattern 
      probabilities estimated by counting alone. Thus the base model provides a 
      smoothing (over the naïve, counting based maximum likelihood estimate) and 
      the weighting with the counting based estimate allows for deviations from 
      this distribution if they are strong enough in the data. The authors 
      discuss several tractable base models (Bernoulli and cascaded logistic) 
      and then proceed to demonstrate that their method optimally captures (when 
      compared to the base models alone and also counting based estimates) the 
      pattern probability distribution for both simulated data and for a 
      population of 10 V1 neurons. I enjoyed reading this paper. It was 
      very clear what the Authors were attempting to do and estimating pattern 
      probability distributions that deviate from standard parametric models 
      (Ising, GLM etc.) is an important goal. The formalism itself seems to be 
      rather standard (very similar to Chapter 25 of Murphy’s Machine Learning 
      book) but I think the application is new, or at least I’ve never seen 
      anyone apply such methods to neural data before. In this vein, I would 
      have liked to have seen more application to real data … but I do 
      understand the space constraints of a NIPS paper. If the authors can, in 
      the final version, include another real data example I think that would 
      make their paper more appealing … but I won’t insist upon this, its just a 
      recommendation. I don’t have any major corrections to the paper. I 
      would recommend some minor additions to the introduction or discussion (a 
      sentence or two would suffice) motivating what the Author’s method is 
      useful for. If I understand the paper correctly, it seems to me that the 
      main utility of this work is in identifying the existence of deviations 
      from the base model as opposed to explaining the source of those 
      deviations. Parametric base models have explanatory power, for example a 
      well-fit Ising model indicates that second order correlations are 
      sufficient to describe the population’s spiking statistics. The Dirichlet 
      process, being non-parametric, really doesn’t tell you why the 
      distribution deviates from the base model … just that it does. If I’m 
      correct here, I think the authors should put a sentence or two in the 
      discussion about these points. There are also a couple typos so a 
      proofreading would be good. The most notable is that the last sentence of 
      the introduction ends mid sentence. In summary, while the 
      mathematical formalism isn’t new, the application is novel and 
      interesting. I think the paper will be of use to the computational 
      neuroscience community and should be accepted as a NIPS paper subject to 
      some minor edits and proofreading.