TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
Despite recent research advancements in reconstructing clothed humans from a
single image, accurately restoring the "unseen regions" with high-level details
remains an unsolved challenge that lacks attention. Existing methods often
generate overly smooth back-side surfaces with a blurry texture. But how to
effectively capture all visual attributes of an individual from a single image,
which are sufficient to reconstruct unseen areas (e.g., the back view)?
Motivated by the power of foundation models, TeCH reconstructs the 3D human by
leveraging 1) descriptive text prompts (e.g., garments, colors, hairstyles)
which are automatically generated via a garment parsing model and Visual
Question Answering (VQA), 2) a personalized fine-tuned Text-to-Image diffusion
model (T2I) which learns the "indescribable" appearance. To represent
high-resolution 3D clothed humans at an affordable cost, we propose a hybrid 3D
representation based on DMTet, which consists of an explicit body shape grid
and an implicit distance field. Guided by the descriptive prompts +
personalized T2I diffusion model, the geometry and texture of the 3D humans are
optimized through multi-view Score Distillation Sampling (SDS) and
reconstruction losses based on the original observation. TeCH produces
high-fidelity 3D clothed humans with consistent & delicate texture, and
detailed full-body geometry. Quantitative and qualitative experiments
demonstrate that TeCH outperforms the state-of-the-art methods in terms of
reconstruction accuracy and rendering quality. The code will be publicly
available for research purposes at https://huangyangyi.github.io/tech
64ddc4a778aef6e489189115
Yangyi Huang
64ddc4a778aef6e489189116
Hongwei Yi
64ddc4a778aef6e489189117
Yuliang Xiu
claimed_verified
2023-08-17T13:53:05.007Z
6183d0b249ef1d984699e4a3
https://cdn-avatars.huggingface.co/v1/production/uploads/6183d0b249ef1d984699e4a3/M6B9b73v8su9H8jfdcbgu.jpeg
Yuliang Xiu
user
Yuliang
64ddc4a778aef6e489189118
Tingting Liao
admin_assigned
2023-08-17T14:01:06.056Z
63e4943b789dcaae43cb5bb5
/avatars/66859007330ddfec09f235e9846d477a.svg
Tingting Liao
user
Luffuly
64ddc4a778aef6e489189119
Jiaxiang Tang
64ddc4a778aef6e48918911a
Deng Cai
admin_assigned
2023-08-17T14:01:37.632Z
646735a1946476c5d214e5b2
/avatars/23392fda23fa6288fcb309f7eaa73472.svg
Deng Cai
user
jcyk
64ddc4a778aef6e48918911b
Justus Thies
TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
Despite recent research advancements in reconstructing clothed humans from a
single image, accurately restoring the "unseen regions" with high-level details
remains an unsolved challenge that lacks attention. Existing methods often
generate overly smooth back-side surfaces with a blurry texture. But how to
effectively capture all visual attributes of an individual from a single image,
which are sufficient to reconstruct unseen areas (e.g., the back view)?
Motivated by the power of foundation models, TeCH reconstructs the 3D human by
leveraging 1) descriptive text prompts (e.g., garments, colors, hairstyles)
which are automatically generated via a garment parsing model and Visual
Question Answering (VQA), 2) a personalized fine-tuned Text-to-Image diffusion
model (T2I) which learns the "indescribable" appearance. To represent
high-resolution 3D clothed humans at an affordable cost, we propose a hybrid 3D
representation based on DMTet, which consists of an explicit body shape grid
and an implicit distance field. Guided by the descriptive prompts +
personalized T2I diffusion model, the geometry and texture of the 3D humans are
optimized through multi-view Score Distillation Sampling (SDS) and
reconstruction losses based on the original observation. TeCH produces
high-fidelity 3D clothed humans with consistent & delicate texture, and
detailed full-body geometry. Quantitative and qualitative experiments
demonstrate that TeCH outperforms the state-of-the-art methods in terms of
reconstruction accuracy and rendering quality. The code will be publicly
available for research purposes at https://huangyangyi.github.io/tech
TeCH reconstructs high-fidelity 3D human models from single images using descriptive text prompts, a personalized Text-to-Image diffusion model, and a hybrid DMTet representation, outperforming existing methods.
descriptive text prompts garment parsing model Visual Question Answering (VQA) Text-to-Image diffusion model DMTet explicit body shape grid implicit distance field multi-view Score Distillation Sampling (SDS) reconstruction losses high-fidelity 3D consistent & delicate texture detailed full-body geometry