Fashion-VDM: Video Diffusion Model for Virtual Try-On
We present Fashion-VDM, a video diffusion model (VDM) for generating virtual
try-on videos. Given an input garment image and person video, our method aims
to generate a high-quality try-on video of the person wearing the given
garment, while preserving the person's identity and motion. Image-based virtual
try-on has shown impressive results; however, existing video virtual try-on
(VVT) methods are still lacking garment details and temporal consistency. To
address these issues, we propose a diffusion-based architecture for video
virtual try-on, split classifier-free guidance for increased control over the
conditioning inputs, and a progressive temporal training strategy for
single-pass 64-frame, 512px video generation. We also demonstrate the
effectiveness of joint image-video training for video try-on, especially when
video data is limited. Our qualitative and quantitative experiments show that
our approach sets the new state-of-the-art for video virtual try-on. For
additional results, visit our project page:
https://johannakarras.github.io/Fashion-VDM.
67290abb74956794331a9ede
Johanna Karras
claimed_verified
2024-11-04T21:24:13.479Z
63334429518c0f78a6e19d65
/avatars/5ac96f0308529cb526cc45030b46b550.svg
Johanna Karras
user
jskarras
67290abb74956794331a9edf
Yingwei Li
67290abb74956794331a9ee0
Nan Liu
claimed_verified
2024-11-05T07:58:34.153Z
62db5ae193fc0e52427a803a
/avatars/251e209b60178a2ffc555b2ecdd29cf1.svg
Nan Liu
user
nanliu
67290abb74956794331a9ee1
Luyang Zhu
67290abb74956794331a9ee2
Innfarn Yoo
67290abb74956794331a9ee3
Andreas Lugmayr
67290abb74956794331a9ee4
Chris Lee
67290abb74956794331a9ee5
Ira Kemelmacher-Shlizerman
Fashion-VDM: Video Diffusion Model for Virtual Try-On
We present Fashion-VDM, a video diffusion model (VDM) for generating virtual
try-on videos. Given an input garment image and person video, our method aims
to generate a high-quality try-on video of the person wearing the given
garment, while preserving the person's identity and motion. Image-based virtual
try-on has shown impressive results; however, existing video virtual try-on
(VVT) methods are still lacking garment details and temporal consistency. To
address these issues, we propose a diffusion-based architecture for video
virtual try-on, split classifier-free guidance for increased control over the
conditioning inputs, and a progressive temporal training strategy for
single-pass 64-frame, 512px video generation. We also demonstrate the
effectiveness of joint image-video training for video try-on, especially when
video data is limited. Our qualitative and quantitative experiments show that
our approach sets the new state-of-the-art for video virtual try-on. For
additional results, visit our project page:
https://johannakarras.github.io/Fashion-VDM.
Fashion-VDM uses a diffusion-based architecture to generate high-quality, temporally consistent, and identity-preserving video try-on videos, addressing issues of detail and temporal consistency in existing methods.
video diffusion model VDM virtual try-on person video garment image split classifier-free guidance progressive temporal training strategy single-pass 64-frame 512px video generation joint image-video training