Deep Convolutional Neural Network for Image Deconvolution
Many fundamental image-related problems involve deconvolution operators. Real blur degradation seldom complies with an ideal linear convolution model due to camera noise, saturation, image compression, to name a few. Instead of perfectly modeling outliers, which is rather challenging from a generative model perspective, we develop a deep convolutional neural network to capture the characteristics of degradation. We note directly applying existing deep neural networks does not produce reasonable results. Our solution is to establish the connection between traditional optimization-based schemes and a neural network architecture where a novel, separable structure is introduced as a reliable support for robust deconvolution against artifacts. Our network contains two submodules, both trained in a supervised manner with proper initialization. They yield decent performance on non-blind image deconvolution compared to previous generative-model based methods.
This paper presents a method for nonblind deconvolution of blurry images, that also can also fix artifacts (e.g. compression, clipping) in the input, and is robust to deviations from the input generation model.  A convolutional network is used both to deblur and fix artifacts; deblurring is performed using a sequence of horizontal and vertical conv kernels, taking advantage of a high degree of separability in the pseudoinverse blur  kernel, and are initialized with a decomposition of the pseudoinverse.  A standard compact-kernel convnet is stacked on top, allowing further fixing of artifacts and noise, and traned end-to-end with pairs of blurry and ground truth images.This is a nice approach that appears to obtain good results on the examples presented.  However, I feel the data could be better explained, and perhaps the test set expanded in size.  I didn't see any description of the sources for the training or testing data -- where did these images come from?  How were the blurry versions generated (in both train or test), and what kernels, noise, etc. were used exactly?  I feel this is a significant gap, and hope it might be addressed in the authors' rebuttal.Questions/comments:* l.150 "by taking a great number of pictures"  -- I'm not sure I understand this, could you explain a bit more?* l.166 "two million sharp patches"  -- from where were these drawn?* l.198: description of figure does not match the figure itself (there is just one example instead of two and the letters do not match)* l.214: extending with large conv kernels:  An alternative might be an FFT conv -- maybe this could be touched on in the text a bit?* l.262:  Is the nonlinearity sigma applied between horizontal and vertical conv layers?  It seems that it is.  If so, the operations are starting to differ from a linear decomposition of the original pseudoinverse convolution.  Is this good or bad in this case?  What if this nonlinearity is removed?* If there is a nonlinearity between horizontal and vertical, is there any benefit to stacking another layer of horizontal and vertical convs (with nonlin), and tweak the initialization either to stretch the pseudoinverse over all these applications or make the second set of kernels start from random?* l.268 "high-dimensional structure make the network more expressive than traditional pseudo-inverse and reasonably robust to outliers":  Which high dimensions are you referring to here?  Is it the 36 maps from the head of the SVD?  If so, would response of the largest eigenvectors potentially be truncated by the sigmoid?  Or does the network learn to work around this by moving away from the decomposition, and this is what makes it robust?* Likewise, I'm curious what might happen by initializing by duplicating the largest eigenvectors multiple times and adding a bit of gaussian noise to the weights?* l.306 "16 images"  -- this seems a bit small for a test set, though I'm not sure how large the images are; perhaps this could be augmented and/or explained a bit more?* l.342 "done by merging" kernels:  The merged kernel is not quite the same network as performing each in sequence:  there are more params.  But this probably is a benefit, allowing the network to perform even better, I suspect.* l.364 "This is the first attempt to make use of generative solvers to guide neural network learning" -- I don't think this is true, and think this sentence probably should be removed.  E.g. naive bayes corresponds to logistic regression; sparse coding informed the design of learned ISTA "Learning Fast Approximations of Sparse Coding" Gregor 2010; unrolled mean-field inference of RBMs in "Multi-prediction deep Boltzmann machines" Godfellow 2013.* sec 7:  Was the network retrained for each blur kernel in the experiments?