title: Garment3DGen: 3D Garment Stylization and Texture Generation
summary: We introduce Garment3DGen a new method to synthesize 3D garment assets from a
  base mesh given a single input image as guidance. Our proposed approach allows
  users to generate 3D textured clothes based on both real and synthetic images,
  such as those generated by text prompts. The generated assets can be directly
  draped and simulated on human bodies. First, we leverage the recent progress of
  image to 3D diffusion methods to generate 3D garment geometries. However, since
  these geometries cannot be utilized directly for downstream tasks, we propose
  to use them as pseudo ground-truth and set up a mesh deformation optimization
  procedure that deforms a base template mesh to match the generated 3D target.
  Second, we introduce carefully designed losses that allow the input base mesh
  to freely deform towards the desired target, yet preserve mesh quality and
  topology such that they can be simulated. Finally, a texture estimation module
  generates high-fidelity texture maps that are globally and locally consistent
  and faithfully capture the input guidance, allowing us to render the generated
  3D assets. With Garment3DGen users can generate the textured 3D garment of
  their choice without the need of artist intervention. One can provide a textual
  prompt describing the garment they desire to generate a simulation-ready 3D
  asset. We present a plethora of quantitative and qualitative comparisons on
  various assets both real and generated and provide use-cases of how one can
  generate simulation-ready 3D garments.
paper_authors._id: 6604e1d41eb292d6df3856a1
paper_authors.name: Nikolaos Sarafianos
paper_authors._id: 6604e1d41eb292d6df3856a2
paper_authors.name: Tuur Stuyck
paper_authors._id: 6604e1d41eb292d6df3856a3
paper_authors.name: Xiaoyu Xiang
paper_authors._id: 6604e1d41eb292d6df3856a4
paper_authors.name: Yilei Li
paper_authors._id: 6604e1d41eb292d6df3856a5
paper_authors.name: Jovan Popovic
paper_authors._id: 6604e1d41eb292d6df3856a6
paper_authors.name: Rakesh Ranjan
paper_title: Garment3DGen: 3D Garment Stylization and Texture Generation
paper_summary: We introduce Garment3DGen a new method to synthesize 3D garment assets from a
  base mesh given a single input image as guidance. Our proposed approach allows
  users to generate 3D textured clothes based on both real and synthetic images,
  such as those generated by text prompts. The generated assets can be directly
  draped and simulated on human bodies. First, we leverage the recent progress of
  image to 3D diffusion methods to generate 3D garment geometries. However, since
  these geometries cannot be utilized directly for downstream tasks, we propose
  to use them as pseudo ground-truth and set up a mesh deformation optimization
  procedure that deforms a base template mesh to match the generated 3D target.
  Second, we introduce carefully designed losses that allow the input base mesh
  to freely deform towards the desired target, yet preserve mesh quality and
  topology such that they can be simulated. Finally, a texture estimation module
  generates high-fidelity texture maps that are globally and locally consistent
  and faithfully capture the input guidance, allowing us to render the generated
  3D assets. With Garment3DGen users can generate the textured 3D garment of
  their choice without the need of artist intervention. One can provide a textual
  prompt describing the garment they desire to generate a simulation-ready 3D
  asset. We present a plethora of quantitative and qualitative comparisons on
  various assets both real and generated and provide use-cases of how one can
  generate simulation-ready 3D garments.
paper_ai_summary: Garment3DGen synthesizes 3D textile assets from 2D images using diffusion methods and mesh deformation, enabling simulation-ready garments from real or generated images, including textual prompts.
paper_ai_keywords: image to 3D diffusion methods mesh deformation optimization pseudo ground-truth mesh quality topology preservation texture estimation high-fidelity texture maps quantitative comparisons qualitative comparisons