title: Controllable Human Image Generation with Personalized Multi-Garments
summary: We present BootComp, a novel framework based on text-to-image diffusion
  models for controllable human image generation with multiple reference
  garments. Here, the main bottleneck is data acquisition for training:
  collecting a large-scale dataset of high-quality reference garment images per
  human subject is quite challenging, i.e., ideally, one needs to manually gather
  every single garment photograph worn by each human. To address this, we propose
  a data generation pipeline to construct a large synthetic dataset, consisting
  of human and multiple-garment pairs, by introducing a model to extract any
  reference garment images from each human image. To ensure data quality, we also
  propose a filtering strategy to remove undesirable generated data based on
  measuring perceptual similarities between the garment presented in human image
  and extracted garment. Finally, by utilizing the constructed synthetic dataset,
  we train a diffusion model having two parallel denoising paths that use
  multiple garment images as conditions to generate human images while preserving
  their fine-grained details. We further show the wide-applicability of our
  framework by adapting it to different types of reference-based generation in
  the fashion domain, including virtual try-on, and controllable human image
  generation with other conditions, e.g., pose, face, etc.
paper_authors._id: 67470ec05535474406e7bede
paper_authors.name: Yisol Choi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-27T22:19:43.181Z
paper_authors._id: 64e23361607e093241e3c271
paper_authors.avatarUrl: /avatars/2a1daf0fee29302c470cffa8e74a0fdb.svg
paper_authors.fullname: Yisol Choi
paper_authors.type: user
paper_authors.user: yisol
paper_authors._id: 67470ec05535474406e7bedf
paper_authors.name: Sangkyung Kwak
paper_authors._id: 67470ec05535474406e7bee0
paper_authors.name: Sihyun Yu
paper_authors._id: 67470ec05535474406e7bee1
paper_authors.name: Hyungwon Choi
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-27T12:21:26.273Z
paper_authors._id: 651f8790559843c34d3d2c22
paper_authors.avatarUrl: /avatars/cbc4189247ed39b761f7c8ffe91d75a7.svg
paper_authors.fullname: Hyungwon Choi
paper_authors.type: user
paper_authors.user: hyungwonc
paper_authors._id: 67470ec05535474406e7bee2
paper_authors.name: Jinwoo Shin
paper_title: Controllable Human Image Generation with Personalized Multi-Garments
paper_summary: We present BootComp, a novel framework based on text-to-image diffusion
  models for controllable human image generation with multiple reference
  garments. Here, the main bottleneck is data acquisition for training:
  collecting a large-scale dataset of high-quality reference garment images per
  human subject is quite challenging, i.e., ideally, one needs to manually gather
  every single garment photograph worn by each human. To address this, we propose
  a data generation pipeline to construct a large synthetic dataset, consisting
  of human and multiple-garment pairs, by introducing a model to extract any
  reference garment images from each human image. To ensure data quality, we also
  propose a filtering strategy to remove undesirable generated data based on
  measuring perceptual similarities between the garment presented in human image
  and extracted garment. Finally, by utilizing the constructed synthetic dataset,
  we train a diffusion model having two parallel denoising paths that use
  multiple garment images as conditions to generate human images while preserving
  their fine-grained details. We further show the wide-applicability of our
  framework by adapting it to different types of reference-based generation in
  the fashion domain, including virtual try-on, and controllable human image
  generation with other conditions, e.g., pose, face, etc.
paper_ai_summary: BootComp uses text-to-image diffusion models to generate human images with multiple garments, overcoming data acquisition challenges through a synthetic dataset and dual denoising paths.
paper_ai_keywords: text-to-image diffusion models data generation pipeline denoising paths perceptual similarities synthetic dataset virtual try-on controllable human image generation