Bayesian Inference for Structured Spike and Slab Priors
Sparse signal recovery addresses the problem of solving underdetermined linear inverse problems subject to a sparsity constraint. We propose a novel prior formulation, the structured spike and slab prior, which allows to incorporate a priori knowledge of the sparsity pattern by imposing a spatial Gaussian process on the spike and slab probabilities. Thus, prior information on the structure of the sparsity pattern can be encoded using generic covariance functions. Furthermore, we provide a Bayesian inference scheme for the proposed model based on the expectation propagation framework. Using numerical experiments on synthetic data, we demonstrate the benefits of the model.
This paper introduces a new spike and slab sparse prior, where the sparse variables are allowed to be correlated and this structure is encoded by generic covariance functions. Therefore, this is a new spike and slab prior for modeling structured sparsity patterns. The authors also propose an algorithm for inference of spike and slab variables under the proposed prior, by applying the framework of expectation propagation. Approximations to a covariance matrix inverse are also proposed, in order to reduce the computational complexity of the algorithm. Numerical results demonstrate that the method significantly outperforms prior art for problems of sparse recovery that exhibit such form of structured sparsity.This paper is an original contribution to the field of sparse recovery and proposes a solution to a significant problem of inference for structured sparsity, one that has been a challenge for many researchers in the last decade. Although the paper does not solve the problem for any general joint distribution of sparse variables, it proposes a solution for a large set of distributions. Therefore, I think it will be of interest for a larger audience and probably useful to many researchers in the field.The paper is clearly written, though notation can be improved (see below for detailed suggestions). Acronyms are not always defined, for example MMV.One improvement I would like to see in the revised paper (besides detailed comments below) is the addition of real data examples where the structure in the sparse variables can be modeled by the set of distributions considered in the paper. The example given in the current version, with EEG measurement matrix, is not convincing since the sparse variables are still sampled from a distribution defined by the authors. I would not call it a real example, since the only real data is the measurement matrix and its properties are not really the target of the paper. Having such examples would make the paper much stronger and the proposed method would have a larger impact.Another point that would be good to discuss in the paper is the attenuation of correlation of sparse variables with respect to the correlation of parameters gamma (Fig. 1c). What does the attenuation depend on? Is there a way to control that dependence? How does the curvature of curves in Fig.1c change with other parameters? I believe this will be an important point to consider in real applications. Detailed comments:- Sec. 2 title: typo - should be "slab" instead of "slap"- line 093: "easy to generate" instead of "easy the generate"- Sec.3. When replacing f1 with f1 tilde, it would be good to explain the switch from the conditional distribution of y given x to a conditional distribution of x given m1 and V1 (where m1 depends on y)- Sec.3. It is not clear where the function d(x,y) is used in 18-21- Sec.3.1. Notation tilde(f)_a,i should be explained, previously was only used tilde(f)_a and now another subscript was added. Maybe it would be better to put i in the superscript? Or just define it in the text- Using bold A and regular A for two different variables is confusing, please consider modifying one.- Sec. 3.2: What is N? Rank of V1?- line 306: "slab" instead of "slap"