Video-Driven Animation of Neural Head Avatars
<|reference_start|>Video-Driven Animation of Neural Head Avatars: We present a new approach for video-driven animation of high-quality neural 3D head models, addressing the challenge of person-independent animation from video input. Typically, high-quality generative models are learned for specific individuals from multi-view video footage, resulting in person-specific latent representations that drive the generation process. In order to achieve person-independent animation from video input, we introduce an LSTM-based animation network capable of translating person-independent expression features into personalized animation parameters of person-specific 3D head models. Our approach combines the advantages of personalized head models (high quality and realism) with the convenience of video-driven animation employing multi-person facial performance capture. We demonstrate the effectiveness of our approach on synthesized animations with high quality based on different source videos as well as an ablation study.<|reference_end|>
@article{paier2024video-driven,
    title={Video-Driven Animation of Neural Head Avatars},
    author={Wolfgang Paier and Paul Hinzer and Anna Hilsmann and Peter Eisert},
    journal={Proc. International Workshop on Vision, Modeling, and
  Visualization (VMV), Braunschweig, Germany, Sep. 2023},
    year={2024},
    doi={10.2312/vmv.20231237},
    archivePrefix={arXiv},
    eprint={2403.04380},
    primaryClass={cs.CV}
}