Multimodal Conditional 3D Face Geometry Generation
<|reference_start|>Multimodal Conditional 3D Face Geometry Generation: We present a new method for multimodal conditional 3D face geometry generation that allows user-friendly control over the output identity and expression via a number of different conditioning signals. Within a single model, we demonstrate 3D faces generated from artistic sketches, 2D face landmarks, Canny edges, FLAME face model parameters, portrait photos, or text prompts. Our approach is based on a diffusion process that generates 3D geometry in a 2D parameterized UV domain. Geometry generation passes each conditioning signal through a set of cross-attention layers (IP-Adapter), one set for each user-defined conditioning signal. The result is an easy-to-use 3D face generation tool that produces high resolution geometry with fine-grain user control.<|reference_end|>
@article{otto2024multimodal,
    title={Multimodal Conditional 3D Face Geometry Generation},
    author={Christopher Otto, Prashanth Chandran, Sebastian Weiss, Markus Gross,
  Gaspard Zoss, Derek Bradley},
    journal={arXiv preprint arXiv:2407.01074},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.01074},
    primaryClass={cs.CV cs.GR}
}