SEEAvatar: Photorealistic Text-to-3D Avatar Generation with Constrained
  Geometry and Appearance
Powered by large-scale text-to-image generation models, text-to-3D avatar
generation has made promising progress. However, most methods fail to produce
photorealistic results, limited by imprecise geometry and low-quality
appearance. Towards more practical avatar generation, we present SEEAvatar, a
method for generating photorealistic 3D avatars from text with SElf-Evolving
constraints for decoupled geometry and appearance. For geometry, we propose to
constrain the optimized avatar in a decent global shape with a template avatar.
The template avatar is initialized with human prior and can be updated by the
optimized avatar periodically as an evolving template, which enables more
flexible shape generation. Besides, the geometry is also constrained by the
static human prior in local parts like face and hands to maintain the delicate
structures. For appearance generation, we use diffusion model enhanced by
prompt engineering to guide a physically based rendering pipeline to generate
realistic textures. The lightness constraint is applied on the albedo texture
to suppress incorrect lighting effect. Experiments show that our method
outperforms previous methods on both global and local geometry and appearance
quality by a large margin. Since our method can produce high-quality meshes and
textures, such assets can be directly applied in classic graphics pipeline for
realistic rendering under any lighting condition. Project page at:
https://seeavatar3d.github.io.
657bd22264dab2f4c9fbfb55
Yuanyou Xu
657bd22264dab2f4c9fbfb56
Zongxin Yang
extracted_pending
2023-12-15T04:12:19.909Z
619bf9b3cbedb87e1a92fb3b
/avatars/ee280db0232e21416c948ab9a9a2344e.svg
Zongxin Yang
user
z-x-yang
657bd22264dab2f4c9fbfb57
Yi Yang
SEEAvatar: Photorealistic Text-to-3D Avatar Generation with Constrained
  Geometry and Appearance
Powered by large-scale text-to-image generation models, text-to-3D avatar
generation has made promising progress. However, most methods fail to produce
photorealistic results, limited by imprecise geometry and low-quality
appearance. Towards more practical avatar generation, we present SEEAvatar, a
method for generating photorealistic 3D avatars from text with SElf-Evolving
constraints for decoupled geometry and appearance. For geometry, we propose to
constrain the optimized avatar in a decent global shape with a template avatar.
The template avatar is initialized with human prior and can be updated by the
optimized avatar periodically as an evolving template, which enables more
flexible shape generation. Besides, the geometry is also constrained by the
static human prior in local parts like face and hands to maintain the delicate
structures. For appearance generation, we use diffusion model enhanced by
prompt engineering to guide a physically based rendering pipeline to generate
realistic textures. The lightness constraint is applied on the albedo texture
to suppress incorrect lighting effect. Experiments show that our method
outperforms previous methods on both global and local geometry and appearance
quality by a large margin. Since our method can produce high-quality meshes and
textures, such assets can be directly applied in classic graphics pipeline for
realistic rendering under any lighting condition. Project page at:
https://seeavatar3d.github.io.
SEEAvatar generates photorealistic 3D avatars from text using a self-evolving template for geometry and a diffusion model-enhanced rendering pipeline for appearance.
SEEAvatar SElf-Evolving constraints template avatar human prior physically based rendering pipeline diffusion model prompt engineering albedo texture lightness constraint