End to End Image Super Resolution via Deep and Shallow Convolutional Networks
In this paper, we propose a new image super resolution (SR) approach based on a convolutional neural network (CNN) which jointly learns the feature extraction, upsampling, and high resolution (HR) reconstruction modules, yielding a completely end to end trainable deep CNN. However, directly training such a deep network in an end to end fashion is challenging, which takes a longer time to converge and may lead to sub optimal results. To address this issue, we propose to jointly train an ensemble of deep and shallow networks. The shallow network with weaker learning capability restores the main structure of the image content, while the deep network with stronger representation power captures the high frequency details. Since the shallow network is much easier to optimize, it significantly lowers the difficulty of deep network optimization during joint training. To further ensure more accurate restoration of HR images, the high frequency details are reconstructed in a multi scale manner to simultaneously incorporate both short and long range contextual information. The proposed method is extensively evaluated on widely adopted data sets and compares favorably against state of the art methods. In depth ablation studies are conducted to verify the contributions of different network designs to image SR, providing additional insights for future research.
IEEE Access