title: Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D
    Diffusion Probabilistic Models
summary: We propose a 3D generation pipeline that uses diffusion models to generate
  realistic human digital avatars. Due to the wide variety of human identities,
  poses, and stochastic details, the generation of 3D human meshes has been a
  challenging problem. To address this, we decompose the problem into 2D normal
  map generation and normal map-based 3D reconstruction. Specifically, we first
  simultaneously generate realistic normal maps for the front and backside of a
  clothed human, dubbed dual normal maps, using a pose-conditional diffusion
  model. For 3D reconstruction, we ``carve'' the prior SMPL-X mesh to a detailed
  3D mesh according to the normal maps through mesh optimization. To further
  enhance the high-frequency details, we present a diffusion resampling scheme on
  both body and facial regions, thus encouraging the generation of realistic
  digital avatars. We also seamlessly incorporate a recent text-to-image
  diffusion model to support text-based human identity control. Our method,
  namely, Chupa, is capable of generating realistic 3D clothed humans with better
  perceptual quality and identity variety.
paper_authors._id: 646ac4954c1cd18b49798325
paper_authors.name: Byungjun Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-22T11:57:29.675Z
paper_authors._id: 644f822623d7eb05ca691d95
paper_authors.avatarUrl: /avatars/b9d3f39155ee96a79b8c873e4bab57e6.svg
paper_authors.fullname: Byungjun Kim
paper_authors.type: user
paper_authors.user: byungjun-kim
paper_authors._id: 646ac4954c1cd18b49798326
paper_authors.name: Patrick Kwon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-22T11:57:26.952Z
paper_authors._id: 63044b2e1dd5d3c624882d19
paper_authors.avatarUrl: /avatars/ba4d387547d1d0baeca918caea680f89.svg
paper_authors.fullname: Patrick Kwon
paper_authors.type: user
paper_authors.user: yj7082126
paper_authors._id: 646ac4954c1cd18b49798327
paper_authors.name: Kwangho Lee
paper_authors._id: 646ac4954c1cd18b49798328
paper_authors.name: Myunggi Lee
paper_authors._id: 646ac4954c1cd18b49798329
paper_authors.name: Sookwan Han
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-27T10:41:01.594Z
paper_authors._id: 628667494713da0266f4d998
paper_authors.avatarUrl: /avatars/64736ed1ad48ee75e3cf53e78b4b622b.svg
paper_authors.fullname: Sookwan Han
paper_authors.type: user
paper_authors.user: jellyheadnadrew
paper_authors._id: 646ac4954c1cd18b4979832a
paper_authors.name: Daesik Kim
paper_authors._id: 646ac4954c1cd18b4979832b
paper_authors.name: Hanbyul Joo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-24T12:19:09.513Z
paper_authors._id: 65b0ebe1165e69e0e6dc6ddd
paper_authors.avatarUrl: /avatars/d2e95b2040c761200df1293db2949da7.svg
paper_authors.fullname: Hanbyul Joo
paper_authors.type: user
paper_authors.user: jhugestar
paper_title: Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D
    Diffusion Probabilistic Models
paper_summary: We propose a 3D generation pipeline that uses diffusion models to generate
  realistic human digital avatars. Due to the wide variety of human identities,
  poses, and stochastic details, the generation of 3D human meshes has been a
  challenging problem. To address this, we decompose the problem into 2D normal
  map generation and normal map-based 3D reconstruction. Specifically, we first
  simultaneously generate realistic normal maps for the front and backside of a
  clothed human, dubbed dual normal maps, using a pose-conditional diffusion
  model. For 3D reconstruction, we ``carve'' the prior SMPL-X mesh to a detailed
  3D mesh according to the normal maps through mesh optimization. To further
  enhance the high-frequency details, we present a diffusion resampling scheme on
  both body and facial regions, thus encouraging the generation of realistic
  digital avatars. We also seamlessly incorporate a recent text-to-image
  diffusion model to support text-based human identity control. Our method,
  namely, Chupa, is capable of generating realistic 3D clothed humans with better
  perceptual quality and identity variety.
paper_ai_summary: A 3D avatar generation pipeline utilizes diffusion models for dual normal map creation and 3D reconstruction, enhanced with text-based identity control.
paper_ai_keywords: diffusion models 3D human meshes 2D normal map generation normal map-based 3D reconstruction dual normal maps pose-conditional diffusion model SMPL-X mesh mesh optimization diffusion resampling scheme text-to-image diffusion model