On Poisson Graphical Models
Undirected graphical models, such as Gaussian graphical models, Ising, and multinomial/categorical graphical models, are widely used in a variety of applications for modeling distributions over a large number of variables. These standard instances, however, are ill-suited to modeling count data, which are increasingly ubiquitous in big-data settings such as genomic sequencing data, user-ratings data, spatial incidence data, climate studies, and site visits. Existing classes of Poisson graphical models, which arise as the joint distributions that correspond to Poisson distributed node-conditional distributions, have a major drawback: they can only model negative conditional dependencies for reasons of normalizability given its infinite domain. In this paper, our objective is to modify the Poisson graphical model distribution so that it can capture a rich dependence structure between count-valued variables. We begin by discussing two strategies for truncating the Poisson distribution and show that only one of these leads to a valid joint distribution. While this model can accommodate a wider range of conditional dependencies, some limitations still remain. To address this, we investigate two additional novel variants of the Poisson distribution and their corresponding joint graphical model distributions. Our three novel approaches provide classes of Poisson-like graphical models that can capture both positive and negative conditional dependencies between count-valued variables. One can learn the graph structure of our models via penalized neighborhood selection, and we demonstrate the performance of our methods by learning simulated networks as well as a network from microRNA-sequencing data.
The paper provides a construction for multivariate 
      distributions over unbounded counts that obey the Markov structure of 
      an undirected network. Building a multivariate distribution 
      over unbounded counts is in general a hard problem, as studied at 
      length by Besag and others for the past 30 years (including textbooks 
      such as Arnold et al.'s "Conditional Specification of Statistical 
      Models", 1999, which extends some of the observations of the authors 
      to other distributions such as conditionally-specified exponential 
      distributions). In the end, the proposal given by the authors 
      succeeds in some relevant ways. The upsides are constructions that do 
      allow for marginal distributions over counts and which lead to 
      relatively simple estimation algorithms. The downsides are, QPGM has 
      marginally thinner tails and SPGM does not have closed-form 
      conditional distributions (which somehow defeats the point of building 
      a conditionally specified model). As a matter of fact, I don't even 
      know how SPGM can be called a Poisson distribution (for QPGM at least 
      one can claim that ``only'' the base measure is being changed). That's 
      OK, but it made me wonder what the main motivation for modeling counts 
      is, since the Poisson itself is not a good distribution to fit 
      empirical data anyway (don't get me wrong, the Poisson is a very 
      useful as a building block to many models - components of stochastic 
      processes and within latent variable models etc. - but could you plot 
      your data for breast cancer and tell me whether it looks anything 
      close to a Poisson?). It would be very useful to have a plot of the 
      probability mass function of the SPGM too, which feels somewhat 
      convoluted at first sight. I suppose you are considering R and R0 as 
      constants (or otherwise these wouldn't be exponential families). How 
      are they chosen? Which advice do you give to the practitioner? 
      That being said: to construct a multivariate distribution over 
      counts obeying the independence model of MRF is hard. I honestly 
      appreciate the effort put in this paper and I think the results are of 
      theoretical interest to NIPS. The only thing that rubs me in the wrong 
      way is the somewhat overly light appreciation of the literature. For 
      instance, it almost feels like the authors don't really know what a 
      copula is. The authors seem not to understand [8] (or at least 
      definitely presented it in the wrong way), for instance, where the 
      whole point is to build multivariate distributions for arbitrary 
      discrete data (count data, inclusive), and for which a battery of MCMC 
      and approximate inference methods exist. It made me wonder whether the 
      discussion of [11] truly makes any sense, since the whole point of 
      that pioneering book is to show how to build discrete models with 
      log-linear parameters in a way it doesn't grow exponentially with the 
      number of variables (although fair enough I don't have a copy of it 
      with me right now and I don't remember anymore what it says about 
      Poisson distributions). But perhaps the worst omission is a complete 
      neglect of the vast spatial statistics literature, where 
      high-dimensional count data analysis has been done for a long time. 
      It needs to be said that several of these approaches (including 
      the sparse precision Gaussian copula model of [8]) don't really model 
      MRF-style independence constraints in the observable space. So as I 
      said the theoretical contribution of this paper is a valid one. But as 
      a practitioner I'm not yet convinced why I should pay the price of 
      sticking to this model space instead of just using the simpler 
      structured Gaussian random field + Poisson measurement model, which 
      has been the standard for a long time. Final comment: I'm not 
      an expert in gene expression analysis at all, but I would be grateful 
      to have a reference newer than [20] claiming that ``counts of 
      sequencing reads ... are replacing microarrays''.