From Bayesian Sparsity to Gated Recurrent Nets
The iterations of many first-order algorithms, when applied to minimizing common regularized regression functions, often resemble neural network layers with prespecified weights. This observation has prompted the development of learningbased approaches that purport to replace these iterations with enhanced surrogates forged as DNN models from available training data. For example, important NPhard sparse estimation problems have recently benefitted from this genre of upgrade, with simple feedforward or recurrent networks ousting proximal gradient-based iterations. Analogously, this paper demonstrates that more powerful Bayesian algorithms for promoting sparsity, which rely on complex multi-loop majorizationminimization techniques, mirror the structure of more sophisticated long short-term memory (LSTM) networks, or alternative gated feedback networks previously designed for sequence prediction. As part of this development, we examine the parallels between latent variable trajectories operating across multiple time-scales during optimization, and the activations within deep network structures designed to adaptively model such characteristic sequences. The resulting insights lead to a novel sparse estimation system that, when granted training data, can estimate optimal solutions efficiently in regimes where other algorithms fail, including practical direction-of-arrival (DOA) and 3D geometry recovery problems. The underlying principles we expose are also suggestive of a learning process for a richer class of multi-loop algorithms in other domains.
This paper proposes a recurrent network for sparse estimation inspired on sparse Bayesian learning (SBL). It first shows that a recurrent architecture can implement a variant of SBL, showing through simulations that different quantities have different time dynamics, which motivates leveraging ideas from recurrent-network design to adapt the architecture. This leads to a recurrent network that seems to outperform approaches based on optimization in simulations and two applications. 
Strengths: The idea of learning a recurrent network for sparse estimation has great potential impact. The paper is very well written. The authors motivate their design decisions in detail and report numerical experiments that are quite thorough and indicate that the technique is successful for challenging sparse-decomposition problems. 
Weaknesses: A lot of the details about the implementation of the method and the experiments are deferred to the supplementary material, so that the main paper is a bit vague. I find this understandable due to length limitations.