Plug-in Estimation in High-Dimensional Linear Inverse Problems: A Rigorous Analysis
Estimating a vector x from noisy linear measurements Ax + w often requires use of prior knowledge or structural constraints on x for accurate reconstruction. Several recent works have considered combining linear least-squares estimation with a generic or "plug-in" denoiser function that can be designed in a modular manner based on the prior knowledge about x. While these methods have shown excellent performance, it has been difficult to obtain rigorous performance guarantees. This work considers plug-in denoising combined with the recentlydeveloped Vector Approximate Message Passing (VAMP) algorithm, which is itself derived via Expectation Propagation techniques. It shown that the mean squared error of this "plug-and-play" VAMP can be exactly predicted for highdimensional right-rotationally invariant random A and Lipschitz denoisers. The method is demonstrated on applications in image recovery and parametric bilinear estimation.
Update: After reading the rebuttal of the authors, I think the authors have addressed my comments satisfactorily. I've increased my score of the paper to 7.  ---------------------------------- Original comments:  Summary:   This paper provides a proof for the state evolution of plug-in denosing-based VAMP algorithm. The results extend the analyses in a previous paper by a subset of the authors. It is well written and the numerical results look nice.  Here are my detailed comments:  1. Is it fair to say that the main difference between the current paper and [21] is related to the “convergent under Gaussian noise” assumption (in Definition 2), and other parts of the proof are parallel to those of [21]?  2. The main contribution of the paper is a proof of state evolution under certain conditions. The authors then provide simulations to show the performance advantage of the proposed algorithm over other algorithms. However, in my view, there seem to be some disconnections between theory and simulations: (1) In the analysis, it is assumed that the denoiser is Lipschitz continuous. However, the denoisers under study (such as the BM3D denoiser and the rank one AMP denoiser for bilinear estimation) doesn’t seem to satisfy such continuity assumptions. The possible discontinuity can have an impact on how you compute the derivatives required by the algorithm. This issue is discussed in [10] and a Monte Carlo method was proposed to address the discontinuity. I’m wondering what’s the impact of discontinuity on the performance of the proposed algorithm? Did you use similar ideas?  (2) It is assumed that $A$ is right rotationally invariant. However, the matrixes involved in the numerical results (particularly the one in the bilinear estimation example) are quite structured. How does such structure affect the accuracy of the state evolution analysis?   (3) The main contribution of the paper is a state evolution theory under certain assumptions. However, the simulations are carried out under conditions (seem (1) (2) above) different from the analysis. It’s not clear whether the theory can still provide good prediction of the empirical performance. To make the story complete, instead of comparing the empirical performance of the algorithm relative to others, I think it’s better to add numerical results that verify the SE predictions.   3. In theorem 1, it is assumed that $\bar{\alpha_{1k}}\in(0,1)$. This assumption seems quite strong. How can one check whether it holds or not? For example, does it hold for the denoisers involved in the numerical results of this paper?  4. The assumption in (13) seem to be extremely strong: it assumes that the initial estimate is an observation of the true signal correction by iid Gaussian noise. Is this practical?  5. The authors cite both [18] and [21]. Sometimes the authors refer to [18] and sometimes to [21]. But it seems that [18] is basically a short version of [21]. It is necessary to include both references?