title: TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
summary: Despite recent research advancements in reconstructing clothed humans from a
  single image, accurately restoring the "unseen regions" with high-level details
  remains an unsolved challenge that lacks attention. Existing methods often
  generate overly smooth back-side surfaces with a blurry texture. But how to
  effectively capture all visual attributes of an individual from a single image,
  which are sufficient to reconstruct unseen areas (e.g., the back view)?
  Motivated by the power of foundation models, TeCH reconstructs the 3D human by
  leveraging 1) descriptive text prompts (e.g., garments, colors, hairstyles)
  which are automatically generated via a garment parsing model and Visual
  Question Answering (VQA), 2) a personalized fine-tuned Text-to-Image diffusion
  model (T2I) which learns the "indescribable" appearance. To represent
  high-resolution 3D clothed humans at an affordable cost, we propose a hybrid 3D
  representation based on DMTet, which consists of an explicit body shape grid
  and an implicit distance field. Guided by the descriptive prompts +
  personalized T2I diffusion model, the geometry and texture of the 3D humans are
  optimized through multi-view Score Distillation Sampling (SDS) and
  reconstruction losses based on the original observation. TeCH produces
  high-fidelity 3D clothed humans with consistent & delicate texture, and
  detailed full-body geometry. Quantitative and qualitative experiments
  demonstrate that TeCH outperforms the state-of-the-art methods in terms of
  reconstruction accuracy and rendering quality. The code will be publicly
  available for research purposes at https://huangyangyi.github.io/tech
paper_authors._id: 64ddc4a778aef6e489189115
paper_authors.name: Yangyi Huang
paper_authors._id: 64ddc4a778aef6e489189116
paper_authors.name: Hongwei Yi
paper_authors._id: 64ddc4a778aef6e489189117
paper_authors.name: Yuliang Xiu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-17T13:53:05.007Z
paper_authors._id: 6183d0b249ef1d984699e4a3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6183d0b249ef1d984699e4a3/GYR4kzPUEJPxrnQD-trhT.png
paper_authors.fullname: Yuliang Xiu
paper_authors.type: user
paper_authors.user: Yuliang
paper_authors._id: 64ddc4a778aef6e489189118
paper_authors.name: Tingting Liao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-17T14:01:06.056Z
paper_authors._id: 63e4943b789dcaae43cb5bb5
paper_authors.avatarUrl: /avatars/66859007330ddfec09f235e9846d477a.svg
paper_authors.fullname: Tingting Liao
paper_authors.type: user
paper_authors.user: Luffuly
paper_authors._id: 64ddc4a778aef6e489189119
paper_authors.name: Jiaxiang Tang
paper_authors._id: 64ddc4a778aef6e48918911a
paper_authors.name: Deng Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-17T14:01:37.632Z
paper_authors._id: 646735a1946476c5d214e5b2
paper_authors.avatarUrl: /avatars/23392fda23fa6288fcb309f7eaa73472.svg
paper_authors.fullname: Deng Cai
paper_authors.type: user
paper_authors.user: jcyk
paper_authors._id: 64ddc4a778aef6e48918911b
paper_authors.name: Justus Thies
paper_title: TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
paper_summary: Despite recent research advancements in reconstructing clothed humans from a
  single image, accurately restoring the "unseen regions" with high-level details
  remains an unsolved challenge that lacks attention. Existing methods often
  generate overly smooth back-side surfaces with a blurry texture. But how to
  effectively capture all visual attributes of an individual from a single image,
  which are sufficient to reconstruct unseen areas (e.g., the back view)?
  Motivated by the power of foundation models, TeCH reconstructs the 3D human by
  leveraging 1) descriptive text prompts (e.g., garments, colors, hairstyles)
  which are automatically generated via a garment parsing model and Visual
  Question Answering (VQA), 2) a personalized fine-tuned Text-to-Image diffusion
  model (T2I) which learns the "indescribable" appearance. To represent
  high-resolution 3D clothed humans at an affordable cost, we propose a hybrid 3D
  representation based on DMTet, which consists of an explicit body shape grid
  and an implicit distance field. Guided by the descriptive prompts +
  personalized T2I diffusion model, the geometry and texture of the 3D humans are
  optimized through multi-view Score Distillation Sampling (SDS) and
  reconstruction losses based on the original observation. TeCH produces
  high-fidelity 3D clothed humans with consistent & delicate texture, and
  detailed full-body geometry. Quantitative and qualitative experiments
  demonstrate that TeCH outperforms the state-of-the-art methods in terms of
  reconstruction accuracy and rendering quality. The code will be publicly
  available for research purposes at https://huangyangyi.github.io/tech
paper_ai_keywords: descriptive text prompts