Text-Guided 3D Face Synthesis -- From Generation to Editing
Text-guided 3D face synthesis has achieved remarkable results by leveraging
text-to-image (T2I) diffusion models. However, most existing works focus solely
on the direct generation, ignoring the editing, restricting them from
synthesizing customized 3D faces through iterative adjustments. In this paper,
we propose a unified text-guided framework from face generation to editing. In
the generation stage, we propose a geometry-texture decoupled generation to
mitigate the loss of geometric details caused by coupling. Besides, decoupling
enables us to utilize the generated geometry as a condition for texture
generation, yielding highly geometry-texture aligned results. We further employ
a fine-tuned texture diffusion model to enhance texture quality in both RGB and
YUV space. In the editing stage, we first employ a pre-trained diffusion model
to update facial geometry or texture based on the texts. To enable sequential
editing, we introduce a UV domain consistency preservation regularization,
preventing unintentional changes to irrelevant facial attributes. Besides, we
propose a self-guided consistency weight strategy to improve editing efficacy
while preserving consistency. Through comprehensive experiments, we showcase
our method's superiority in face synthesis. Project page:
https://faceg2e.github.io/.
656d3cccefd0eea7c5cd3c60
Yunjie Wu
claimed_verified
2023-12-04T09:35:56.117Z
6461d6135e410d3ff8ec3e25
/avatars/0fd09e1cc1b75751dc7f22e9c166d8b5.svg
yunjie wu
user
jiejaingwu
656d3cccefd0eea7c5cd3c61
Yapeng Meng
656d3cccefd0eea7c5cd3c62
Zhipeng Hu
admin_assigned
2023-12-04T10:47:04.323Z
63427b63e2647466b4252c03
/avatars/1a9e069c56852fa976578d9538159e92.svg
hu zhipeng
user
hufuzhipeng
656d3cccefd0eea7c5cd3c63
Lincheng Li
656d3cccefd0eea7c5cd3c64
Haoqian Wu
admin_assigned
2023-12-04T10:47:12.886Z
64dc932a968fb479b20dffdf
/avatars/1b7a574d6f57433509f56b5bf2b66644.svg
Haoqian Wu
user
woolseyyy
656d3cccefd0eea7c5cd3c65
Kun Zhou
656d3cccefd0eea7c5cd3c66
Weiwei Xu
admin_assigned
2023-12-04T10:49:29.548Z
64d461bce1830780115da214
/avatars/5864b19347911504e84af8c27186bf9d.svg
Weiwei Xu
user
xuww
656d3cccefd0eea7c5cd3c67
Xin Yu
Text-Guided 3D Face Synthesis -- From Generation to Editing
Text-guided 3D face synthesis has achieved remarkable results by leveraging
text-to-image (T2I) diffusion models. However, most existing works focus solely
on the direct generation, ignoring the editing, restricting them from
synthesizing customized 3D faces through iterative adjustments. In this paper,
we propose a unified text-guided framework from face generation to editing. In
the generation stage, we propose a geometry-texture decoupled generation to
mitigate the loss of geometric details caused by coupling. Besides, decoupling
enables us to utilize the generated geometry as a condition for texture
generation, yielding highly geometry-texture aligned results. We further employ
a fine-tuned texture diffusion model to enhance texture quality in both RGB and
YUV space. In the editing stage, we first employ a pre-trained diffusion model
to update facial geometry or texture based on the texts. To enable sequential
editing, we introduce a UV domain consistency preservation regularization,
preventing unintentional changes to irrelevant facial attributes. Besides, we
propose a self-guided consistency weight strategy to improve editing efficacy
while preserving consistency. Through comprehensive experiments, we showcase
our method's superiority in face synthesis. Project page:
https://faceg2e.github.io/.
A text-guided framework for 3D face synthesis and editing uses decoupled geometry-texture generation and fine-tuned texture diffusion to achieve high quality and consistent results.
diffusion models text-to-image (T2I) diffusion models geometry-texture decoupled generation UV domain consistency preservation regularization self-guided consistency weight strategy texture diffusion model