Distilling Generative-Discriminative Representations for Very Low-Resolution Face Recognition
<|reference_start|>Distilling Generative-Discriminative Representations for Very Low-Resolution Face Recognition: Very low-resolution face recognition is challenging due to the serious loss of informative facial details in resolution degradation. In this paper, we propose a generative-discriminative representation distillation approach that combines generative representation with cross-resolution aligned knowledge distillation. This approach facilitates very low-resolution face recognition by jointly distilling generative and discriminative models via two distillation modules. Firstly, the generative representation distillation takes the encoder of a diffusion model pretrained for face super-resolution as the generative teacher to supervise the learning of the student backbone via feature regression, and then freezes the student backbone. After that, the discriminative representation distillation further considers a pretrained face recognizer as the discriminative teacher to supervise the learning of the student head via cross-resolution relational contrastive distillation. In this way, the general backbone representation can be transformed into discriminative head representation, leading to a robust and discriminative student model for very low-resolution face recognition. Our approach improves the recovery of the missing details in very low-resolution faces and achieves better knowledge transfer. Extensive experiments on face datasets demonstrate that our approach enhances the recognition accuracy of very low-resolution faces, showcasing its effectiveness and adaptability.<|reference_end|>
@article{zhang2024distilling,
    title={Distilling Generative-Discriminative Representations for Very
  Low-Resolution Face Recognition},
    author={Junzheng Zhang, Weijia Guo, Bochao Liu, Ruixin Shi, Yong Li and
  Shiming Ge},
    journal={arXiv preprint arXiv:2409.06371},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.06371},
    primaryClass={cs.CV cs.AI cs.MM}
}