title: HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
summary: Current advances in human head modeling allow to generate plausible-looking
  3D head models via neural representations. Nevertheless, constructing complete
  high-fidelity head models with explicitly controlled animation remains an
  issue. Furthermore, completing the head geometry based on a partial
  observation, e.g. coming from a depth sensor, while preserving details is often
  problematic for the existing methods. We introduce a generative model for
  detailed 3D head meshes on top of an articulated 3DMM which allows explicit
  animation and high-detail preservation at the same time. Our method is trained
  in two stages. First, we register a parametric head model with vertex
  displacements to each mesh of the recently introduced NPHM dataset of accurate
  3D head scans. The estimated displacements are baked into a hand-crafted UV
  layout. Second, we train a StyleGAN model in order to generalize over the UV
  maps of displacements. The decomposition of the parametric model and
  high-quality vertex displacements allows us to animate the model and modify it
  semantically. We demonstrate the results of unconditional generation and
  fitting to the full or partial observation. The project page is available at
  https://seva100.github.io/headcraft.
paper_authors._id: 658506c4c76915aa14cc8bb8
paper_authors.name: Artem Sevastopolsky
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-22T14:28:59.317Z
paper_authors._id: 64638395d2044cd1d7c7e832
paper_authors.avatarUrl: /avatars/6541363767ba7a91b59e68c400817376.svg
paper_authors.fullname: Artem Sevastopolskiy
paper_authors.type: user
paper_authors.user: seva100
paper_authors._id: 658506c4c76915aa14cc8bb9
paper_authors.name: Philip-William Grassal
paper_authors._id: 658506c4c76915aa14cc8bba
paper_authors.name: Simon Giebenhain
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T12:34:05.641Z
paper_authors._id: 64dc981b96fbcd6b19f687b5
paper_authors.avatarUrl: /avatars/6eca33e3fcca750d4f6e9b0f7d724d8e.svg
paper_authors.fullname: Giebenhain
paper_authors.type: user
paper_authors.user: SimonGiebenhain
paper_authors._id: 658506c4c76915aa14cc8bbb
paper_authors.name: ShahRukh Athar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T12:34:13.916Z
paper_authors._id: 650c3a2bd439bbbbadf40436
paper_authors.avatarUrl: /avatars/a5a7c04a03580ae59c28eb6f90d43b47.svg
paper_authors.fullname: ShahRukh Athar
paper_authors.type: user
paper_authors.user: Sathar
paper_authors._id: 658506c4c76915aa14cc8bbc
paper_authors.name: Luisa Verdoliva
paper_authors._id: 658506c4c76915aa14cc8bbd
paper_authors.name: Matthias Niessner
paper_title: HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
paper_summary: Current advances in human head modeling allow to generate plausible-looking
  3D head models via neural representations. Nevertheless, constructing complete
  high-fidelity head models with explicitly controlled animation remains an
  issue. Furthermore, completing the head geometry based on a partial
  observation, e.g. coming from a depth sensor, while preserving details is often
  problematic for the existing methods. We introduce a generative model for
  detailed 3D head meshes on top of an articulated 3DMM which allows explicit
  animation and high-detail preservation at the same time. Our method is trained
  in two stages. First, we register a parametric head model with vertex
  displacements to each mesh of the recently introduced NPHM dataset of accurate
  3D head scans. The estimated displacements are baked into a hand-crafted UV
  layout. Second, we train a StyleGAN model in order to generalize over the UV
  maps of displacements. The decomposition of the parametric model and
  high-quality vertex displacements allows us to animate the model and modify it
  semantically. We demonstrate the results of unconditional generation and
  fitting to the full or partial observation. The project page is available at
  https://seva100.github.io/headcraft.
paper_ai_keywords: neural representations 3D head models generative model 3DMM (3D Morphable Model) StyleGAN UV layout level stability unconditional generation fitting partial observation