Designed Measurements for Vector Count Data
We consider design of linear projection measurements for a vector Poisson signal model. The projections are performed on the vector Poisson rate, X ∈ R n + , and the observed data are a vector of counts, Y ∈ Z m + . The projection matrix is designed by maximizing mutual information between Y and X, I(Y ; X). When there is a latent class label C ∈ {1, . . . , L} associated with X, we consider the mutual information with respect to Y and C, I(Y ; C). New analytic expressions for the gradient of I(Y ; X) and I(Y ; C) are presented, with gradient performed with respect to the measurement matrix. Connections are made to the more widely studied Gaussian measurement model. Example results are presented for compressive topic modeling of a document corpora (word counting), and hyperspectral compressive sensing for chemical classification (photon counting).
The paper proposes a design for measurement matrices 
      of for compressed sensing for the case when the signal being measured is a 
      Poisson vector. They propose selecting the design matrix to optimize for 
      the mutual information I(X; Y) between the sensed signal (X) and the 
      measurement (Y), which could be used for signal reconstruction. They also 
      consider the design matrix chosen to optimize the mutual information I(Y; 
      C) between the measurement and the underlying class (label) that generates 
      the signal X. This could be used for classification purposes. They derive 
      closed form expressions of I(Y; X) and I(Y; C), and use these to generate 
      gradients. Their work is an extension of the work by Palomar/Verdu for the 
      Gaussian case. The main contribution theoretical contribution of 
      the paper is the derivation of the gradient of the mutual information 
      (which leads to an algorithm for selection of the design matrix). However, 
      the quality of the resulting solution is not characterized. The 
      experimental results are also a little weak . The chemical 
      sensing/classification problem, which seems to be the more interesting 
      one, only uses 5 measurements - it would have been at least interesting to 
      consider more measurements (at least for the simulation, because the graph 
      shows the designed system to be saturating, while the accuracy of the 
      randomly chosen matrix keeps increasing). Especially given that full 
      reconstruction is often the goal, it would have been interesting to know 
      the minimum number of measurements that are required to achieve the same 
      quality as the full information case (both for the random and the designed 
      matrices). The results from the 20 newsgroup dataset does not seem like a 
      particularly good application either - the system as described is not 
      better than the state of the art, and it is disappointing that the 
      improvement obtained by using a model which looks like it takes 
      significant amount of work to derive and compute generates roughly the 
      same accruracy as one using twice the number of random measurements (in 
      other words, using the vastly simpler random sensing matrix, which makes 
      no assumption about the underlying model (and so is likely more robust), 
      we need about 2x the number of measurements to achieve the same level of 
      accuracy as a specifically designed model). The 2x factor could certainly 
      be significant in a number of cases, but it is unclear if this is one of 
      them. However, I think that this is a reasonable result to publish 
      (perhaps simply to give people a better sense about the limits of designed 
      sensing matrices). In). While the extensions to the Poisson model are not 
      technically challenging, I found them interesting, especially the chemical 
      sensing application. The connection between information theory and MMSE 
      estimation is interesting. I dont think that the Palomar/Verdu result, 
      connecting MMSE to the gradient of th emutaul information is a widely 
      known result in the NIPS community (based on my unscientific poll of 5 
      people at least), and so its use and extension is likely a worthwhile 
      addition to NIPS. The paper is fairly clear (though many of 
      the important details, such as the definition of the regularity conditions 
      needed for their results has been shunted off to the supplementary 
      material). The presentation of experimental results is a little sloppy 
      though (the graphs are in color and are hard to read on a printed copy; 
      the labels on the confusion matrix seem to be incorrect (the there are too 
      many row labels in Figure 3(b) for example; it is pretty normal to use the 
      same order in the rows and columns, which the authors dont do).