  In this study, we introduce a feature knowledge distillation framework to
improve low-resolution (LR) face recognition performance using knowledge
obtained from high-resolution (HR) images. The proposed framework transfers
informative features from an HR-trained network to an LR-trained network by
reducing the distance between them. A cosine similarity measure was employed as
a distance metric to effectively align the HR and LR features. This approach
differs from conventional knowledge distillation frameworks, which use the L_p
distance metrics and offer the advantage of converging well when reducing the
distance between features of different resolutions. Our framework achieved a 3%
improvement over the previous state-of-the-art method on the AgeDB-30 benchmark
without bells and whistles, while maintaining a strong performance on HR
images. The effectiveness of cosine similarity as a distance metric was
validated through statistical analysis, making our approach a promising
solution for real-world applications in which LR images are frequently
encountered. The code and pretrained models are publicly available on
https://github.com/gist-ailab/feature-similarity-KD.

Computer Vision and Pattern Recognition