Controllable Human Image Generation with Personalized Multi-Garments
We present BootComp, a novel framework based on text-to-image diffusion
models for controllable human image generation with multiple reference
garments. Here, the main bottleneck is data acquisition for training:
collecting a large-scale dataset of high-quality reference garment images per
human subject is quite challenging, i.e., ideally, one needs to manually gather
every single garment photograph worn by each human. To address this, we propose
a data generation pipeline to construct a large synthetic dataset, consisting
of human and multiple-garment pairs, by introducing a model to extract any
reference garment images from each human image. To ensure data quality, we also
propose a filtering strategy to remove undesirable generated data based on
measuring perceptual similarities between the garment presented in human image
and extracted garment. Finally, by utilizing the constructed synthetic dataset,
we train a diffusion model having two parallel denoising paths that use
multiple garment images as conditions to generate human images while preserving
their fine-grained details. We further show the wide-applicability of our
framework by adapting it to different types of reference-based generation in
the fashion domain, including virtual try-on, and controllable human image
generation with other conditions, e.g., pose, face, etc.
67470ec05535474406e7bede
Yisol Choi
claimed_verified
2024-11-27T22:19:43.181Z
64e23361607e093241e3c271
/avatars/2a1daf0fee29302c470cffa8e74a0fdb.svg
Yisol Choi
user
yisol
67470ec05535474406e7bedf
Sangkyung Kwak
67470ec05535474406e7bee0
Sihyun Yu
67470ec05535474406e7bee1
Hyungwon Choi
extracted_pending
2024-11-27T12:21:26.273Z
651f8790559843c34d3d2c22
/avatars/cbc4189247ed39b761f7c8ffe91d75a7.svg
Hyungwon Choi
user
hyungwonc
67470ec05535474406e7bee2
Jinwoo Shin
Controllable Human Image Generation with Personalized Multi-Garments
We present BootComp, a novel framework based on text-to-image diffusion
models for controllable human image generation with multiple reference
garments. Here, the main bottleneck is data acquisition for training:
collecting a large-scale dataset of high-quality reference garment images per
human subject is quite challenging, i.e., ideally, one needs to manually gather
every single garment photograph worn by each human. To address this, we propose
a data generation pipeline to construct a large synthetic dataset, consisting
of human and multiple-garment pairs, by introducing a model to extract any
reference garment images from each human image. To ensure data quality, we also
propose a filtering strategy to remove undesirable generated data based on
measuring perceptual similarities between the garment presented in human image
and extracted garment. Finally, by utilizing the constructed synthetic dataset,
we train a diffusion model having two parallel denoising paths that use
multiple garment images as conditions to generate human images while preserving
their fine-grained details. We further show the wide-applicability of our
framework by adapting it to different types of reference-based generation in
the fashion domain, including virtual try-on, and controllable human image
generation with other conditions, e.g., pose, face, etc.
BootComp uses text-to-image diffusion models to generate human images with multiple garments, overcoming data acquisition challenges through a synthetic dataset and dual denoising paths.
text-to-image diffusion models data generation pipeline denoising paths perceptual similarities synthetic dataset virtual try-on controllable human image generation