DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D
  Diffusion
Leveraging pretrained 2D diffusion models and score distillation sampling
(SDS), recent methods have shown promising results for text-to-3D avatar
generation. However, generating high-quality 3D avatars capable of expressive
animation remains challenging. In this work, we present DreamWaltz-G, a novel
learning framework for animatable 3D avatar generation from text. The core of
this framework lies in Skeleton-guided Score Distillation and Hybrid 3D
Gaussian Avatar representation. Specifically, the proposed skeleton-guided
score distillation integrates skeleton controls from 3D human templates into 2D
diffusion models, enhancing the consistency of SDS supervision in terms of view
and human pose. This facilitates the generation of high-quality avatars,
mitigating issues such as multiple faces, extra limbs, and blurring. The
proposed hybrid 3D Gaussian avatar representation builds on the efficient 3D
Gaussians, combining neural implicit fields and parameterized 3D meshes to
enable real-time rendering, stable SDS optimization, and expressive animation.
Extensive experiments demonstrate that DreamWaltz-G is highly effective in
generating and animating 3D avatars, outperforming existing methods in both
visual quality and animation expressiveness. Our framework further supports
diverse applications, including human video reenactment and multi-subject scene
composition.
66f4c54b529e23b71ff7b9f6
Yukun Huang
claimed_verified
2024-09-26T08:23:06.509Z
638ee900ee7e45e0474a5712
https://cdn-avatars.huggingface.co/v1/production/uploads/638ee900ee7e45e0474a5712/KLli_eCbWwffKR7oLDmV3.jpeg
Yukun Huang
user
KevinHuang
66f4c54b529e23b71ff7b9f7
Jianan Wang
66f4c54b529e23b71ff7b9f8
Ailing Zeng
extracted_pending
2024-09-26T02:22:06.392Z
665030b8c08859923b274a55
/avatars/06a3f3cd533caa459f3da1bb5e0f4c1f.svg
aznukad
user
alkxncda
66f4c54b529e23b71ff7b9f9
Zheng-Jun Zha
66f4c54b529e23b71ff7b9fa
Lei Zhang
66f4c54b529e23b71ff7b9fb
Xihui Liu
claimed_verified
2024-10-16T09:44:40.118Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D
  Diffusion
Leveraging pretrained 2D diffusion models and score distillation sampling
(SDS), recent methods have shown promising results for text-to-3D avatar
generation. However, generating high-quality 3D avatars capable of expressive
animation remains challenging. In this work, we present DreamWaltz-G, a novel
learning framework for animatable 3D avatar generation from text. The core of
this framework lies in Skeleton-guided Score Distillation and Hybrid 3D
Gaussian Avatar representation. Specifically, the proposed skeleton-guided
score distillation integrates skeleton controls from 3D human templates into 2D
diffusion models, enhancing the consistency of SDS supervision in terms of view
and human pose. This facilitates the generation of high-quality avatars,
mitigating issues such as multiple faces, extra limbs, and blurring. The
proposed hybrid 3D Gaussian avatar representation builds on the efficient 3D
Gaussians, combining neural implicit fields and parameterized 3D meshes to
enable real-time rendering, stable SDS optimization, and expressive animation.
Extensive experiments demonstrate that DreamWaltz-G is highly effective in
generating and animating 3D avatars, outperforming existing methods in both
visual quality and animation expressiveness. Our framework further supports
diverse applications, including human video reenactment and multi-subject scene
composition.
DreamWaltz-G generates and animates high-quality 3D avatars from text using skeleton-guided score distillation and hybrid 3D Gaussian avatar representation, enhancing consistency and expressiveness.
diffusion models score distillation sampling SDS skeleton-guided score distillation neural implicit fields parameterized 3D meshes hybrid 3D Gaussian avatar representation