Compressed Sensing with Deep Image Prior and Learned Regularization
We propose a novel method for compressed sensing recovery using untrained deep generative models. Our method is based on the recently proposed Deep Image Prior (DIP), wherein the convolutional weights of the network are optimized to match the observed measurements. We show that this approach can be applied to solve any differentiable linear inverse problem, outperforming previous unlearned methods. Unlike various learned approaches based on generative models, our method does not require pre-training over large datasets. We further introduce a novel learned regularization technique, which incorporates prior information on the network weights. This reduces reconstruction error, especially for noisy measurements. Finally we prove that, using the DIP optimization approach, moderately overparameterized single-layer networks trained can perfectly fit any signal despite the nonconvex nature of the fitting problem. This theoretical result provides justification for early stopping.
This paper proposes use of the deep image prior (DIP) in compressed sensing. The proposed method, termed CS-DIP, solves the nonlinear regularized least square regression (equation (3)). It is especially beneficial in that it does not require training using a large-scale dataset if the learned regularization is not used. Results of numerical experiments demonstrate empirical superiority of the proposed method on the reconstruction of chest x-ray images as well as on that of the MNIST handwritten digit images. The demonstrated empirical efficiency of the proposal is itself interesting. At the same time, the proposal can be regarded as a straightforward combination of compressed sensing and the DIP, so that the main contribution of this paper should be considered rather marginal. I would thus like to recommend *weak accept* of this paper. In my view the concept of DIP provides a very stimulating working hypothesis which claims that what is important in image reconstruction is not really representation learning but rather an appropriate network architecture. The results of this paper can be regarded as providing an additional empirical support for this working hypothesis. On the other hand, what we have understood in this regard seem quite limited; for example, on the basis of the contents of this paper one cannot determine what network architecture should be used in the proposed CS-DIP framework applied to a specific task. I think that the fact that DIP is still a working hypothesis should be stressed more in this paper. It does not reduce the value of this paper as one providing an empirical support for it. I think that the theoretical result of this paper, summarized as Theorem 4.1, does not tell us much about CS-DIP. The theorem shows that overfitting occurs even if one uses a single hidden-layer ReLU network. As the authors argue, it would suggest necessity of early stopping in the proposal. On the other hand, I could not find any discussion on early stopping in the experiments: On page 14, lines 22-23, it seems that the theoretical result is not taken into account in deciding the stopping criterion, which would make the significance of the theoretical contribution quite obscure. Page 4, line 10: a phenomen(a -> on) Page 13: The paper by Ulyanov, Vedaldi, and Lempitsky on DIP has been published as a conference paper in the proceedings of CVPR 2018, so that appropriate bibliographic information should be provided. Page 16, line 26: On the right-hand side of the equation, the outermost phi should not be there. Page 17, line 3: 3828 should perhaps read 3528.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Compressed Sensing with Deep Image Prior and Learned Regularization
We propose a novel method for compressed sensing recovery using untrained deep generative models. Our method is based on the recently proposed Deep Image Prior (DIP), wherein the convolutional weights of the network are optimized to match the observed measurements. We show that this approach can be applied to solve any differentiable linear inverse problem, outperforming previous unlearned methods. Unlike various learned approaches based on generative models, our method does not require pre-training over large datasets. We further introduce a novel learned regularization technique, which incorporates prior information on the network weights. This reduces reconstruction error, especially for noisy measurements. Finally we prove that, using the DIP optimization approach, moderately overparameterized single-layer networks trained can perfectly fit any signal despite the nonconvex nature of the fitting problem. This theoretical result provides justification for early stopping.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes use of the deep image prior (DIP) in compressed sensing. The proposed method, termed CS-DIP, solves the nonlinear regularized least square regression (equation (3)). It is especially beneficial in that it does not require training using a large-scale dataset if the learned regularization is not used. Results of numerical experiments demonstrate empirical superiority of the proposed method on the reconstruction of chest x-ray images as well as on that of the MNIST handwritten digit images. The demonstrated empirical efficiency of the proposal is itself interesting. At the same time, the proposal can be regarded as a straightforward combination of compressed sensing and the DIP, so that the main contribution of this paper should be considered rather marginal. I would thus like to recommend *weak accept* of this paper. In my view the concept of DIP provides a very stimulating working hypothesis which claims that what is important in image reconstruction is not really representation learning but rather an appropriate network architecture. The results of this paper can be regarded as providing an additional empirical support for this working hypothesis. On the other hand, what we have understood in this regard seem quite limited; for example, on the basis of the contents of this paper one cannot determine what network architecture should be used in the proposed CS-DIP framework applied to a specific task. I think that the fact that DIP is still a working hypothesis should be stressed more in this paper. It does not reduce the value of this paper as one providing an empirical support for it. I think that the theoretical result of this paper, summarized as Theorem 4.1, does not tell us much about CS-DIP. The theorem shows that overfitting occurs even if one uses a single hidden-layer ReLU network. As the authors argue, it would suggest necessity of early stopping in the proposal. On the other hand, I could not find any discussion on early stopping in the experiments: On page 14, lines 22-23, it seems that the theoretical result is not taken into account in deciding the stopping criterion, which would make the significance of the theoretical contribution quite obscure. Page 4, line 10: a phenomen(a -> on) Page 13: The paper by Ulyanov, Vedaldi, and Lempitsky on DIP has been published as a conference paper in the proceedings of CVPR 2018, so that appropriate bibliographic information should be provided. Page 16, line 26: On the right-hand side of the equation, the outermost phi should not be there. Page 17, line 3: 3828 should perhaps read 3528.
assistant
This paper proposes use of the deep image prior (DIP) in compressed sensing. The proposed method, termed CS-DIP, solves the nonlinear regularized least square regression (equation (3)). It is especially beneficial in that it does not require training using a large-scale dataset if the learned regularization is not used. Results of numerical experiments demonstrate empirical superiority of the proposed method on the reconstruction of chest x-ray images as well as on that of the MNIST handwritten digit images. The demonstrated empirical efficiency of the proposal is itself interesting. At the same time, the proposal can be regarded as a straightforward combination of compressed sensing and the DIP, so that the main contribution of this paper should be considered rather marginal. I would thus like to recommend *weak accept* of this paper. In my view the concept of DIP provides a very stimulating working hypothesis which claims that what is important in image reconstruction is not really representation learning but rather an appropriate network architecture. The results of this paper can be regarded as providing an additional empirical support for this working hypothesis. On the other hand, what we have understood in this regard seem quite limited; for example, on the basis of the contents of this paper one cannot determine what network architecture should be used in the proposed CS-DIP framework applied to a specific task. I think that the fact that DIP is still a working hypothesis should be stressed more in this paper. It does not reduce the value of this paper as one providing an empirical support for it. I think that the theoretical result of this paper, summarized as Theorem 4.1, does not tell us much about CS-DIP. The theorem shows that overfitting occurs even if one uses a single hidden-layer ReLU network. As the authors argue, it would suggest necessity of early stopping in the proposal. On the other hand, I could not find any discussion on early stopping in the experiments: On page 14, lines 22-23, it seems that the theoretical result is not taken into account in deciding the stopping criterion, which would make the significance of the theoretical contribution quite obscure. Page 4, line 10: a phenomen(a -> on) Page 13: The paper by Ulyanov, Vedaldi, and Lempitsky on DIP has been published as a conference paper in the proceedings of CVPR 2018, so that appropriate bibliographic information should be provided. Page 16, line 26: On the right-hand side of the equation, the outermost phi should not be there. Page 17, line 3: 3828 should perhaps read 3528.