title: UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with
    Authenticity Guided Textures
summary: Recent advances in 3D avatar generation have gained significant attentions.
  These breakthroughs aim to produce more realistic animatable avatars, narrowing
  the gap between virtual and real-world experiences. Most of existing works
  employ Score Distillation Sampling (SDS) loss, combined with a differentiable
  renderer and text condition, to guide a diffusion model in generating 3D
  avatars. However, SDS often generates oversmoothed results with few facial
  details, thereby lacking the diversity compared with ancestral sampling. On the
  other hand, other works generate 3D avatar from a single image, where the
  challenges of unwanted lighting effects, perspective views, and inferior image
  quality make them difficult to reliably reconstruct the 3D face meshes with the
  aligned complete textures. In this paper, we propose a novel 3D avatar
  generation approach termed UltrAvatar with enhanced fidelity of geometry, and
  superior quality of physically based rendering (PBR) textures without unwanted
  lighting. To this end, the proposed approach presents a diffuse color
  extraction model and an authenticity guided texture diffusion model. The former
  removes the unwanted lighting effects to reveal true diffuse colors so that the
  generated avatars can be rendered under various lighting conditions. The latter
  follows two gradient-based guidances for generating PBR textures to render
  diverse face-identity features and details better aligning with 3D mesh
  geometry. We demonstrate the effectiveness and robustness of the proposed
  method, outperforming the state-of-the-art methods by a large margin in the
  experiments.
paper_authors._id: 65af569d06916708a9298eba
paper_authors.name: Mingyuan Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-23T11:14:27.971Z
paper_authors._id: 653b2524b77b5e255f2d29d2
paper_authors.avatarUrl: /avatars/f69aea8de84c435295e7638bad5bd82e.svg
paper_authors.fullname: Mingyuan Zhou
paper_authors.type: user
paper_authors.user: mingyuanzhou
paper_authors._id: 65af569d06916708a9298ebb
paper_authors.name: Rakib Hyder
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-23T11:14:40.058Z
paper_authors._id: 6437384ec96e0edeaab5d432
paper_authors.avatarUrl: /avatars/af801b0b86ae62e96d464c0d3e1be57f.svg
paper_authors.fullname: Rakib Hyder
paper_authors.type: user
paper_authors.user: rakibhyder
paper_authors._id: 65af569d06916708a9298ebc
paper_authors.name: Ziwei Xuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-23T11:14:46.514Z
paper_authors._id: 646fa2bc799a974be31a3158
paper_authors.avatarUrl: /avatars/d48453bf9908305765728f9106a4f96a.svg
paper_authors.fullname: Ziwei Xuan
paper_authors.type: user
paper_authors.user: gokyo
paper_authors._id: 65af569d06916708a9298ebd
paper_authors.name: Guojun Qi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-23T11:14:58.973Z
paper_authors._id: 63f3bb014745321de354ea1f
paper_authors.avatarUrl: /avatars/359c7876503f99f1cca9646cc497781d.svg
paper_authors.fullname: qiguojun
paper_authors.type: user
paper_authors.user: Quintin0429
paper_title: UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with
    Authenticity Guided Textures
paper_summary: Recent advances in 3D avatar generation have gained significant attentions.
  These breakthroughs aim to produce more realistic animatable avatars, narrowing
  the gap between virtual and real-world experiences. Most of existing works
  employ Score Distillation Sampling (SDS) loss, combined with a differentiable
  renderer and text condition, to guide a diffusion model in generating 3D
  avatars. However, SDS often generates oversmoothed results with few facial
  details, thereby lacking the diversity compared with ancestral sampling. On the
  other hand, other works generate 3D avatar from a single image, where the
  challenges of unwanted lighting effects, perspective views, and inferior image
  quality make them difficult to reliably reconstruct the 3D face meshes with the
  aligned complete textures. In this paper, we propose a novel 3D avatar
  generation approach termed UltrAvatar with enhanced fidelity of geometry, and
  superior quality of physically based rendering (PBR) textures without unwanted
  lighting. To this end, the proposed approach presents a diffuse color
  extraction model and an authenticity guided texture diffusion model. The former
  removes the unwanted lighting effects to reveal true diffuse colors so that the
  generated avatars can be rendered under various lighting conditions. The latter
  follows two gradient-based guidances for generating PBR textures to render
  diverse face-identity features and details better aligning with 3D mesh
  geometry. We demonstrate the effectiveness and robustness of the proposed
  method, outperforming the state-of-the-art methods by a large margin in the
  experiments.
paper_ai_keywords: Score Distillation Sampling (SDS) differentiable renderer text condition ancestral sampling 3D avatar generation facial details geometry fidelity physically based rendering (PBR) textures diffuse color extraction model texture diffusion model gradient-based guidance