title: ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image
summary: Recent advancements in text-to-image generation have enabled significant
  progress in zero-shot 3D shape generation. This is achieved by score
  distillation, a methodology that uses pre-trained text-to-image diffusion
  models to optimize the parameters of a 3D neural presentation, e.g. Neural
  Radiance Field (NeRF). While showing promising results, existing methods are
  often not able to preserve the geometry of complex shapes, such as human
  bodies. To address this challenge, we present ZeroAvatar, a method that
  introduces the explicit 3D human body prior to the optimization process.
  Specifically, we first estimate and refine the parameters of a parametric human
  body from a single image. Then during optimization, we use the posed parametric
  body as additional geometry constraint to regularize the diffusion model as
  well as the underlying density field. Lastly, we propose a UV-guided texture
  regularization term to further guide the completion of texture on invisible
  body parts. We show that ZeroAvatar significantly enhances the robustness and
  3D consistency of optimization-based image-to-3D avatar generation,
  outperforming existing zero-shot image-to-3D methods.
paper_authors._id: 647412babf9b32c6bbcce340
paper_authors.name: Zhenzhen Weng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-31T07:24:20.475Z
paper_authors._id: 630834fbf0dc38fb47c245b2
paper_authors.avatarUrl: /avatars/579d083fdbfba7331b07941f0bef83a4.svg
paper_authors.fullname: Zhenzhen Weng
paper_authors.type: user
paper_authors.user: zzweng
paper_authors._id: 647412babf9b32c6bbcce341
paper_authors.name: Zeyu Wang
paper_authors._id: 647412babf9b32c6bbcce342
paper_authors.name: Serena Yeung
paper_title: ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image
paper_summary: Recent advancements in text-to-image generation have enabled significant
  progress in zero-shot 3D shape generation. This is achieved by score
  distillation, a methodology that uses pre-trained text-to-image diffusion
  models to optimize the parameters of a 3D neural presentation, e.g. Neural
  Radiance Field (NeRF). While showing promising results, existing methods are
  often not able to preserve the geometry of complex shapes, such as human
  bodies. To address this challenge, we present ZeroAvatar, a method that
  introduces the explicit 3D human body prior to the optimization process.
  Specifically, we first estimate and refine the parameters of a parametric human
  body from a single image. Then during optimization, we use the posed parametric
  body as additional geometry constraint to regularize the diffusion model as
  well as the underlying density field. Lastly, we propose a UV-guided texture
  regularization term to further guide the completion of texture on invisible
  body parts. We show that ZeroAvatar significantly enhances the robustness and
  3D consistency of optimization-based image-to-3D avatar generation,
  outperforming existing zero-shot image-to-3D methods.
paper_ai_keywords: score distillation text-to-image diffusion models Neural Radiance Field (NeRF) 3D neural presentation parametric human body geometry constraint density field UV-guided texture regularization