title: DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D
    Diffusion
summary: Leveraging pretrained 2D diffusion models and score distillation sampling
  (SDS), recent methods have shown promising results for text-to-3D avatar
  generation. However, generating high-quality 3D avatars capable of expressive
  animation remains challenging. In this work, we present DreamWaltz-G, a novel
  learning framework for animatable 3D avatar generation from text. The core of
  this framework lies in Skeleton-guided Score Distillation and Hybrid 3D
  Gaussian Avatar representation. Specifically, the proposed skeleton-guided
  score distillation integrates skeleton controls from 3D human templates into 2D
  diffusion models, enhancing the consistency of SDS supervision in terms of view
  and human pose. This facilitates the generation of high-quality avatars,
  mitigating issues such as multiple faces, extra limbs, and blurring. The
  proposed hybrid 3D Gaussian avatar representation builds on the efficient 3D
  Gaussians, combining neural implicit fields and parameterized 3D meshes to
  enable real-time rendering, stable SDS optimization, and expressive animation.
  Extensive experiments demonstrate that DreamWaltz-G is highly effective in
  generating and animating 3D avatars, outperforming existing methods in both
  visual quality and animation expressiveness. Our framework further supports
  diverse applications, including human video reenactment and multi-subject scene
  composition.
paper_authors._id: 66f4c54b529e23b71ff7b9f6
paper_authors.name: Yukun Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-26T08:23:06.509Z
paper_authors._id: 638ee900ee7e45e0474a5712
paper_authors.avatarUrl: /avatars/eadb5ae2fc92bd9af5516acbc8f1bdf0.svg
paper_authors.fullname: Yukun Huang
paper_authors.type: user
paper_authors.user: KevinHuang
paper_authors._id: 66f4c54b529e23b71ff7b9f7
paper_authors.name: Jianan Wang
paper_authors._id: 66f4c54b529e23b71ff7b9f8
paper_authors.name: Ailing Zeng
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-26T02:22:06.392Z
paper_authors._id: 665030b8c08859923b274a55
paper_authors.avatarUrl: /avatars/06a3f3cd533caa459f3da1bb5e0f4c1f.svg
paper_authors.fullname: aznukad
paper_authors.type: user
paper_authors.user: alkxncda
paper_authors._id: 66f4c54b529e23b71ff7b9f9
paper_authors.name: Zheng-Jun Zha
paper_authors._id: 66f4c54b529e23b71ff7b9fa
paper_authors.name: Lei Zhang
paper_authors._id: 66f4c54b529e23b71ff7b9fb
paper_authors.name: Xihui Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-16T09:44:40.118Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D
    Diffusion
paper_summary: Leveraging pretrained 2D diffusion models and score distillation sampling
  (SDS), recent methods have shown promising results for text-to-3D avatar
  generation. However, generating high-quality 3D avatars capable of expressive
  animation remains challenging. In this work, we present DreamWaltz-G, a novel
  learning framework for animatable 3D avatar generation from text. The core of
  this framework lies in Skeleton-guided Score Distillation and Hybrid 3D
  Gaussian Avatar representation. Specifically, the proposed skeleton-guided
  score distillation integrates skeleton controls from 3D human templates into 2D
  diffusion models, enhancing the consistency of SDS supervision in terms of view
  and human pose. This facilitates the generation of high-quality avatars,
  mitigating issues such as multiple faces, extra limbs, and blurring. The
  proposed hybrid 3D Gaussian avatar representation builds on the efficient 3D
  Gaussians, combining neural implicit fields and parameterized 3D meshes to
  enable real-time rendering, stable SDS optimization, and expressive animation.
  Extensive experiments demonstrate that DreamWaltz-G is highly effective in
  generating and animating 3D avatars, outperforming existing methods in both
  visual quality and animation expressiveness. Our framework further supports
  diverse applications, including human video reenactment and multi-subject scene
  composition.
paper_ai_keywords: diffusion models score distillation sampling (SDS) animatable 3D avatar generation Skeleton-guided Score Distillation Hybrid 3D Gaussian Avatar representation skeleton controls SDS supervision view and human pose multiple faces extra limbs blurring efficient 3D Gaussians neural implicit fields parameterized 3D meshes real-time rendering stable SDS optimization expressive animation human video reenactment multi-subject scene composition