FSRNet: End to End Learning Face Super Resolution with Facial Priors
Face Super Resolution (SR) is a domain specific superresolution problem. The facial prior knowledge can be leveraged to better super resolve face images. We present a novel deep end to end trainable Face Super Resolution Network (FSRNet) which makes use of the geometry prior, i.e. facial landmark heatmaps and parsing maps, to super resolve very low resolution (LR) face images without well aligned requirement. Specifically, we first construct a coarse SR network to recover a coarse high resolution (HR) image. Then, the coarse HR image is sent to two branches: a fine SR encoder and a prior information estimation network, which extracts the image features, and estimates landmark heatmaps/parsing maps respectively. Both image features and prior information are sent to a fine SR decoder to recover the HR image. To generate realistic faces, we also propose the Face Super Resolution Generative Adversarial Network (FSRGAN) to incorporate the adversarial loss into FSRNet. Further, we introduce two related tasks, face alignment and parsing, as the new evaluation metrics for face SR, which address the inconsistency of classic metrics w.r.t. visual perception. Extensive experiments show that FSRNet and FSRGAN significantly outperforms state of the arts for very LR face SR, both quantitatively and qualitatively.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition