ALISTA: ANALYTIC WEIGHTS ARE AS GOOD AS LEARNED WEIGHTS IN LISTA
Deep neural networks based on unfolding an iterative algorithm, for example, LISTA (learned iterative shrinkage thresholding algorithm), have been an empirical success for sparse signal recovery. The weights of these neural networks are currently determined by data-driven "black-box" training. In this work, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is computed as the solution to a data-free optimization problem, leaving only the stepsize and threshold parameters to data-driven learning. This significantly simplifies the training. Specifically, the data-free optimization problem is based on coherence minimization. We show our ALISTA retains the optimal linear convergence proved in (Chen et al., 2018)  and has a performance comparable to LISTA. Furthermore, we extend ALISTA to convolutional linear operators, again determined in a data-free manner. We also propose a feed-forward framework that combines the data-free optimization and ALISTA networks from end to end, one that can be jointly trained to gain robustness to small perturbations in the encoding model.
The papers studies neural network-based sparse signal recovery, and derives many new theoretical insights into the classical LISTA model. The authors proposed Analytic LISTA (ALISTA), where the weight matrix in LISTA is pre-computed with a data-free coherence minimization, followed by a separate data-driven learning step for merely (a very small number of) step-size and threshold parameters. Their theory is extensible to convolutional cases. The two-stage decomposed pipeline was shown to keep the optimal linear convergence proved in (Chen et al., 2018). Experiments observe that ALISTA has almost no performance loss compared to the much heavier parameterized LISTA, in contrast to the common wisdom that (brutal-force) “end-to-end” always outperforms stage-wise training. Their contributions thus manifest in both novel theory results, and the practical impacts of simplifying/accelerating LISTA training.  Besides, they also proposed an interesting new strategy called Robust ALISTA to overcome the small perturbations on the encoding basis, which also benefits from this decomposed problems structure. 
The proofs and conclusions are mathematically correct to my best knowledge. I personally worked on similar sparse unfolding problems before so this work looks particularly novel and interesting to me. My intuition then was that, it should not be really necessary to use heavily parameterized networks to approximate a simple linear sparse coding form (LISTA idea). Similar accelerations could have been achieved with line search for something similar to steepest descent (also computational expensive, but need learn step-sizes only, and agnostic to input distribution). Correspondingly, there should exist a more elegant network solution with very light learnable weights. This work perfectly coincides with the intuition, providing very solid guidance on how a LISTA model could be built right. Given in recent three years, many application works rely on unfold-truncating techniques (compressive sensing, reconstruction, super resolution, image restoration, clustering…), I envision this paper to generate important impacts for practitioners pursuing those ideas. 
Additionally, I like Theorem 3 in Section 3.1, on the provable efficient approximation of general convolution using circular convolution. It could be useful for many other problems such as filter response matching. 
I therefore hold a very positive attitude towards this paper and support for its acceptance. Some questions I would like the authors to clarify & improve in revision:
1.	Eqn (7) assumes noise-free case. The author stated “The zero-noise assumption is for simplicity of the proofs.” Could the authors elaborate which part of current theory/proof will fail in noisy case? If so, can it be overcome (even by less “simpler” way)? How about convolutional case, the same? Could the authors at least provide some empirical results for ALISTA’s performance under noise?
2.	Section 5.3. It is unclear to me why Robust ALISTA has to work better than the data augmented ALISTA. Is it potentially because that in the data augmentation baseline, the training data volume is much amplified, and one ALISTA model might become underfitting? It would be interesting to create a larger-capacity ALISTA model (e.g., by increasing unfolded layer numbers), train it on the augmented data, and see if it can compare more favorably against Robust ALISTA?
3.	The writeup is overall very good, mature, and easy to follow. But still, typos occur from time to time, showing a bit rush. For example, Section 5.1, “the x-axes denotes is the indices of layers” should remove “is”. Please make sure more proofreading will be done.