ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA
Deep neural networks based on unfolding an iterative algorithm, for example, LISTA (learned iterative shrinkage thresholding algorithm), have been an empirical success for sparse signal recovery. The weights of these neural networks are currently determined by data-driven “black-box” training. In this work, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is computed as the solution to a data-free optimization problem, leaving only the stepsize and threshold parameters to data-driven learning. This signiﬁcantly simpliﬁes the training. Speciﬁcally, the data-free optimization problem is based on coherence minimization. We show our ALISTA retains the optimal linear convergence proved in (Chen et al., 2018) and has a performance comparable to LISTA. Furthermore, we extend ALISTA to convolutional linear operators, again determined in a data-free manner. We also propose a feed-forward framework that combines the data-free optimization and ALISTA networks from end to end, one that can be jointly trained to gain robustness to small perturbations in the encoding model.
 The paper raises many important questions about unrolled iterative optimization algorithms, and answers many questions for the case of iterative soft thresholding algorithm (ISTA, and learned variant LISTA). The authors demonstrate that a major simplification is available for the learned network: instead of learning a matrix for each layer, or even a single (potentially large) matrix, one may obtain the matrix analytically and learn only a series of scalars. These simplifications are not only practically useful but allow for theoretical analysis in the context of optimization theory. On top of this seminal contribution, the results are extended to the convolutional-LISTA setting. Finally, yet another fascinating result is presented, namely that the analytic weights can be determined from a Gaussian-perturbed version of the dictionary. Experimental validation of all results is presented. My only constructive criticism of this paper are a few grammatical typos, but specifically the 2nd to last sentence before Sec 2.1 states the wrong thing "In this way, the LISTA model could be further significantly simplified, without little performance loss" ... it should be "with little".
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA
Deep neural networks based on unfolding an iterative algorithm, for example, LISTA (learned iterative shrinkage thresholding algorithm), have been an empirical success for sparse signal recovery. The weights of these neural networks are currently determined by data-driven “black-box” training. In this work, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is computed as the solution to a data-free optimization problem, leaving only the stepsize and threshold parameters to data-driven learning. This signiﬁcantly simpliﬁes the training. Speciﬁcally, the data-free optimization problem is based on coherence minimization. We show our ALISTA retains the optimal linear convergence proved in (Chen et al., 2018) and has a performance comparable to LISTA. Furthermore, we extend ALISTA to convolutional linear operators, again determined in a data-free manner. We also propose a feed-forward framework that combines the data-free optimization and ALISTA networks from end to end, one that can be jointly trained to gain robustness to small perturbations in the encoding model.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper raises many important questions about unrolled iterative optimization algorithms, and answers many questions for the case of iterative soft thresholding algorithm (ISTA, and learned variant LISTA). The authors demonstrate that a major simplification is available for the learned network: instead of learning a matrix for each layer, or even a single (potentially large) matrix, one may obtain the matrix analytically and learn only a series of scalars. These simplifications are not only practically useful but allow for theoretical analysis in the context of optimization theory. On top of this seminal contribution, the results are extended to the convolutional-LISTA setting. Finally, yet another fascinating result is presented, namely that the analytic weights can be determined from a Gaussian-perturbed version of the dictionary. Experimental validation of all results is presented. My only constructive criticism of this paper are a few grammatical typos, but specifically the 2nd to last sentence before Sec 2.1 states the wrong thing "In this way, the LISTA model could be further significantly simplified, without little performance loss" ... it should be "with little".
assistant
 The paper raises many important questions about unrolled iterative optimization algorithms, and answers many questions for the case of iterative soft thresholding algorithm (ISTA, and learned variant LISTA). The authors demonstrate that a major simplification is available for the learned network: instead of learning a matrix for each layer, or even a single (potentially large) matrix, one may obtain the matrix analytically and learn only a series of scalars. These simplifications are not only practically useful but allow for theoretical analysis in the context of optimization theory. On top of this seminal contribution, the results are extended to the convolutional-LISTA setting. Finally, yet another fascinating result is presented, namely that the analytic weights can be determined from a Gaussian-perturbed version of the dictionary. Experimental validation of all results is presented. My only constructive criticism of this paper are a few grammatical typos, but specifically the 2nd to last sentence before Sec 2.1 states the wrong thing "In this way, the LISTA model could be further significantly simplified, without little performance loss" ... it should be "with little".