title: Text-Guided Generation and Editing of Compositional 3D Avatars
summary: Our goal is to create a realistic 3D facial avatar with hair and accessories
  using only a text description. While this challenge has attracted significant
  recent interest, existing methods either lack realism, produce unrealistic
  shapes, or do not support editing, such as modifications to the hairstyle. We
  argue that existing methods are limited because they employ a monolithic
  modeling approach, using a single representation for the head, face, hair, and
  accessories. Our observation is that the hair and face, for example, have very
  different structural qualities that benefit from different representations.
  Building on this insight, we generate avatars with a compositional model, in
  which the head, face, and upper body are represented with traditional 3D
  meshes, and the hair, clothing, and accessories with neural radiance fields
  (NeRF). The model-based mesh representation provides a strong geometric prior
  for the face region, improving realism while enabling editing of the person's
  appearance. By using NeRFs to represent the remaining components, our method is
  able to model and synthesize parts with complex geometry and appearance, such
  as curly hair and fluffy scarves. Our novel system synthesizes these
  high-quality compositional avatars from text descriptions. The experimental
  results demonstrate that our method, Text-guided generation and Editing of
  Compositional Avatars (TECA), produces avatars that are more realistic than
  those of recent methods while being editable because of their compositional
  nature. For example, our TECA enables the seamless transfer of compositional
  features like hairstyles, scarves, and other accessories between avatars. This
  capability supports applications such as virtual try-on.
paper_authors._id: 6502a22ca33ce3ae286cffd9
paper_authors.name: Hao Zhang
paper_authors._id: 6502a22ca33ce3ae286cffda
paper_authors.name: Yao Feng
paper_authors._id: 6502a22ca33ce3ae286cffdb
paper_authors.name: Peter Kulits
paper_authors._id: 6502a22ca33ce3ae286cffdc
paper_authors.name: Yandong Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-14T07:38:39.245Z
paper_authors._id: 643c21735fcffe09fb68a46f
paper_authors.avatarUrl: /avatars/76aabacd318aa954d4c53094ad456056.svg
paper_authors.fullname: Yandong Wen
paper_authors.type: user
paper_authors.user: ydwen
paper_authors._id: 6502a22ca33ce3ae286cffdd
paper_authors.name: Justus Thies
paper_authors._id: 6502a22ca33ce3ae286cffde
paper_authors.name: Michael J. Black
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-14T07:38:50.415Z
paper_authors._id: 649fbababa7a05e3626546c8
paper_authors.avatarUrl: /avatars/a9aa4b4914649527094fc1b12779e014.svg
paper_authors.fullname: Michael J. Black
paper_authors.type: user
paper_authors.user: MichaelJBlack
paper_title: Text-Guided Generation and Editing of Compositional 3D Avatars
paper_summary: Our goal is to create a realistic 3D facial avatar with hair and accessories
  using only a text description. While this challenge has attracted significant
  recent interest, existing methods either lack realism, produce unrealistic
  shapes, or do not support editing, such as modifications to the hairstyle. We
  argue that existing methods are limited because they employ a monolithic
  modeling approach, using a single representation for the head, face, hair, and
  accessories. Our observation is that the hair and face, for example, have very
  different structural qualities that benefit from different representations.
  Building on this insight, we generate avatars with a compositional model, in
  which the head, face, and upper body are represented with traditional 3D
  meshes, and the hair, clothing, and accessories with neural radiance fields
  (NeRF). The model-based mesh representation provides a strong geometric prior
  for the face region, improving realism while enabling editing of the person's
  appearance. By using NeRFs to represent the remaining components, our method is
  able to model and synthesize parts with complex geometry and appearance, such
  as curly hair and fluffy scarves. Our novel system synthesizes these
  high-quality compositional avatars from text descriptions. The experimental
  results demonstrate that our method, Text-guided generation and Editing of
  Compositional Avatars (TECA), produces avatars that are more realistic than
  those of recent methods while being editable because of their compositional
  nature. For example, our TECA enables the seamless transfer of compositional
  features like hairstyles, scarves, and other accessories between avatars. This
  capability supports applications such as virtual try-on.
paper_ai_summary: A system generating realistic 3D avatars from text descriptions using a compositional model with traditional 3D meshes for the face and neural radiance fields (NeRF) for hair, accessories, and clothing.
paper_ai_keywords: 3D meshes neural radiance fields NeRF