SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation
<|reference_start|>SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation: Self-portraits captured from a short distance might look unnatural or even unattractive due to heavy distortions making facial features malformed, and ill-placed head poses. In this paper, we propose SUPER, a novel method of eliminating distortions and adjusting head pose in a close-up face crop. We perform 3D GAN inversion for a facial image by optimizing camera parameters and face latent code, which gives a generated image. Besides, we estimate depth from the obtained latent code, create a depth-induced 3D mesh, and render it with updated camera parameters to obtain a warped portrait. Finally, we apply the visibility-based blending so that visible regions are reprojected, and occluded parts are restored with a generative model. Experiments on face undistortion benchmarks and on our self-collected Head Rotation dataset (HeRo), show that SUPER outperforms previous approaches both qualitatively and quantitatively, opening new possibilities for photorealistic selfie editing.<|reference_end|>
@article{karpikova2024super:,
    title={SUPER: Selfie Undistortion and Head Pose Editing with Identity
  Preservation},
    author={Polina Karpikova, Andrei Spiridonov, Anna Vorontsova, Anastasia
  Yaschenko, Ekaterina Radionova, Igor Medvedev, Alexander Limonov},
    journal={arXiv preprint arXiv:2406.12700},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.12700},
    primaryClass={cs.CV cs.LG}
}