Trained generative models have shown remarkable performance as priors for inverse problems in imaging. For example, Generative Adversarial Network priors permit recovery of test images from 5-10x fewer measurements than sparsity priors. Unfortunately, these models may be unable to represent any particular image because of architectural choices, mode collapse, and bias in the training dataset. In this paper, we demonstrate that invertible neural networks, which have zero representation error by design, can be effective natural signal priors at inverse problems such as denoising, compressive sensing, and inpainting. Our formulation is an empirical risk minimization that does not directly optimize the likelihood of images, as one would expect. Instead we optimize the likelihood of the latent representation of images as a proxy, as this is empirically easier. For compressive sensing, our formulation can yield higher accuracy than sparsity priors across almost all undersampling ratios. For the same accuracy on test images, they can use 10-20x fewer measurements. We demonstrate that invertible priors can yield better reconstructions than sparsity priors for images that have rare features of variation within the biased training set, including out-of-distribution natural images.
Update: I have read the other reviews and the author response and have not changed my evaluation.
Recent work has shown that GANs can be effective for use as priors in inverse problems for images such as compressed sensing, denoising, and inpainting. A drawback is that GANs may have the problem of inexact reconstruction, and strongly reflect the biases in the training set yielding poor performance on out-of-distribution data. This paper shows that the exact inverses available to normalizing flow models and their broad assignment of likelihoods allows for better reconstructions especially on out-of-domain data.
As far as I know, this is the first work to use normalizing flows for inpainting and compressed sensing. The approach and application is very natural, although it’s a bit surprising that using the likelihood as a prior term directly did not work very well. The results of this work show that invertible generative models have utility for inverse image problems even when the quality of raw samples is substantially below GANs. In my opinion the main advantage in this method is not on having low reconstruction error on observed pixels, which becomes less of a problem for more powerful GAN models, but rather the good performance on out of domain data which is somewhat surprising. The authors are reasonably thorough, testing their model on a variety of problem settings and perform ablation studies on hyperparameters.
As additional baselines for compressed sensing and denoising, it would be good to compare to the Deep Image Prior since there is effectively no out-of-distribution input for this untrained model and it performs well with moderate image corruption. Additional discussion about the two could be useful, as for the Deep Image Prior a similar patter is observed where denoising requires explicit regularization (early stopping or gradient noise for DIP) but image completion and compressed sensing do not. Also, there have been many improvements to DCGAN over the years that might ameliorate the problems that were observed in reconstruction, but I don’t fault the authors much for this as it can be difficult training models like StyleGAN even at 64x64 sizes.
It might also be interesting to know whether the good performance on out-of-distribution inputs is due to the exact invertibility or the log-likelihood objective, although I would guess that it is the latter. On way to test this would be training the GLOW model with an adversarial objective instead of NLL as done in [1].
Minor Comments:
Figure 4 would probably be better with a logarithmic scaling for # of measurements
I did not understand the comment about sublevel sets of the data misfit term being inverse images of cylinders, maybe this could use some elaboration.
[1] https://arxiv.org/abs/1705.08868