Real-time 3D-aware Portrait Editing from a Single Image
<|reference_start|>Real-time 3D-aware Portrait Editing from a Single Image: This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our method achieves real-time editing with a feedforward network (i.e., ~0.04s per image), over 100x faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with ~5min fine-tuning per style).<|reference_end|>
@article{bai2024real-time,
    title={Real-time 3D-aware Portrait Editing from a Single Image},
    author={Qingyan Bai, Zifan Shi, Yinghao Xu, Hao Ouyang, Qiuyu Wang, Ceyuan
  Yang, Xuan Wang, Gordon Wetzstein, Yujun Shen, Qifeng Chen},
    journal={arXiv preprint arXiv:2402.14000},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.14000},
    primaryClass={cs.CV}
}