OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable
  Virtual Try-on
Image-based virtual try-on (VTON), which aims to generate an outfitted image
of a target human wearing an in-shop garment, is a challenging image-synthesis
task calling for not only high fidelity of the outfitted human but also full
preservation of garment details. To tackle this issue, we propose Outfitting
over Try-on Diffusion (OOTDiffusion), leveraging the power of pretrained latent
diffusion models and designing a novel network architecture for realistic and
controllable virtual try-on. Without an explicit warping process, we propose an
outfitting UNet to learn the garment detail features, and merge them with the
target human body via our proposed outfitting fusion in the denoising process
of diffusion models. In order to further enhance the controllability of our
outfitting UNet, we introduce outfitting dropout to the training process, which
enables us to adjust the strength of garment features through classifier-free
guidance. Our comprehensive experiments on the VITON-HD and Dress Code datasets
demonstrate that OOTDiffusion efficiently generates high-quality outfitted
images for arbitrary human and garment images, which outperforms other VTON
methods in both fidelity and controllability, indicating an impressive
breakthrough in virtual try-on. Our source code is available at
https://github.com/levihsu/OOTDiffusion.
65e6ccfd26cbe9fbd260d28a
Yuhao Xu
admin_assigned
2024-03-07T09:18:49.387Z
64469dde90b3fae03880344e
https://cdn-avatars.huggingface.co/v1/production/uploads/64469dde90b3fae03880344e/VnLVt_NG3pAjWz0lRoaXm.jpeg
Yuhao Xu
user
levihsu
65e6ccfd26cbe9fbd260d28b
Tao Gu
claimed_verified
2024-03-07T09:15:51.316Z
64d4a72884dc227f34003401
https://cdn-avatars.huggingface.co/v1/production/uploads/64d4a72884dc227f34003401/YCSNTZD7-uQ3tZ_hMC9-M.jpeg
Tao Gu
user
GT2048
65e6ccfd26cbe9fbd260d28c
Weifeng Chen
claimed_verified
2024-03-07T09:15:54.212Z
6489541e182a613250f21426
/avatars/837053b14b35bd3b5c37649166d94374.svg
WEIFENG
user
ShineChen1024
65e6ccfd26cbe9fbd260d28d
Chengcai Chen
OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable
  Virtual Try-on
Image-based virtual try-on (VTON), which aims to generate an outfitted image
of a target human wearing an in-shop garment, is a challenging image-synthesis
task calling for not only high fidelity of the outfitted human but also full
preservation of garment details. To tackle this issue, we propose Outfitting
over Try-on Diffusion (OOTDiffusion), leveraging the power of pretrained latent
diffusion models and designing a novel network architecture for realistic and
controllable virtual try-on. Without an explicit warping process, we propose an
outfitting UNet to learn the garment detail features, and merge them with the
target human body via our proposed outfitting fusion in the denoising process
of diffusion models. In order to further enhance the controllability of our
outfitting UNet, we introduce outfitting dropout to the training process, which
enables us to adjust the strength of garment features through classifier-free
guidance. Our comprehensive experiments on the VITON-HD and Dress Code datasets
demonstrate that OOTDiffusion efficiently generates high-quality outfitted
images for arbitrary human and garment images, which outperforms other VTON
methods in both fidelity and controllability, indicating an impressive
breakthrough in virtual try-on. Our source code is available at
https://github.com/levihsu/OOTDiffusion.
OOTDiffusion uses latent diffusion models and a novel outfitting UNet to generate high-fidelity and controllable virtual try-on images with preserved garment details.
latent diffusion models outfitting UNet denoising process diffusion models outfitting fusion outfitting dropout classifier-free guidance