TryOnDiffusion: A Tale of Two UNets
Given two images depicting a person and a garment worn by another person, our
goal is to generate a visualization of how the garment might look on the input
person. A key challenge is to synthesize a photorealistic detail-preserving
visualization of the garment, while warping the garment to accommodate a
significant body pose and shape change across the subjects. Previous methods
either focus on garment detail preservation without effective pose and shape
variation, or allow try-on with the desired shape and pose but lack garment
details. In this paper, we propose a diffusion-based architecture that unifies
two UNets (referred to as Parallel-UNet), which allows us to preserve garment
details and warp the garment for significant pose and body change in a single
network. The key ideas behind Parallel-UNet include: 1) garment is warped
implicitly via a cross attention mechanism, 2) garment warp and person blend
happen as part of a unified process as opposed to a sequence of two separate
tasks. Experimental results indicate that TryOnDiffusion achieves
state-of-the-art performance both qualitatively and quantitatively.
648bf365f11318e0a448a91c
Luyang Zhu
admin_assigned
2023-06-16T07:06:44.785Z
6426836b13a9e5d9675a6571
/avatars/6b2925901e36fddb95760e3cbd20d8d7.svg
Luyang Zhu
user
lyzhu
648bf365f11318e0a448a91d
Dawei Yang
admin_assigned
2023-06-16T07:07:01.097Z
6480e70d8c6a3b8f11fae86f
/avatars/aab6be8626d9a2a87f58c8a5803c72f0.svg
DaweiYang
user
DaweiYang
648bf365f11318e0a448a91e
Tyler Zhu
648bf365f11318e0a448a91f
Fitsum Reda
claimed_verified
2024-11-03T14:26:27.153Z
672170c49dd8f8814e66a245
https://cdn-avatars.huggingface.co/v1/production/uploads/672170c49dd8f8814e66a245/Ksfb4iJ5AvlF5NAekH5Ww.jpeg
Fitsum Reda
user
fitsumreda
648bf365f11318e0a448a920
William Chan
648bf365f11318e0a448a921
Chitwan Saharia
648bf365f11318e0a448a922
Mohammad Norouzi
648bf365f11318e0a448a923
Ira Kemelmacher-Shlizerman
claimed_verified
2024-09-02T07:20:19.754Z
66d20e411ba71ac4c0488132
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/kkqDAFT15lx4wrJVXiFGJ.png
Ira Kemelmacher-Shlizerman
user
kemelmi
TryOnDiffusion: A Tale of Two UNets
Given two images depicting a person and a garment worn by another person, our
goal is to generate a visualization of how the garment might look on the input
person. A key challenge is to synthesize a photorealistic detail-preserving
visualization of the garment, while warping the garment to accommodate a
significant body pose and shape change across the subjects. Previous methods
either focus on garment detail preservation without effective pose and shape
variation, or allow try-on with the desired shape and pose but lack garment
details. In this paper, we propose a diffusion-based architecture that unifies
two UNets (referred to as Parallel-UNet), which allows us to preserve garment
details and warp the garment for significant pose and body change in a single
network. The key ideas behind Parallel-UNet include: 1) garment is warped
implicitly via a cross attention mechanism, 2) garment warp and person blend
happen as part of a unified process as opposed to a sequence of two separate
tasks. Experimental results indicate that TryOnDiffusion achieves
state-of-the-art performance both qualitatively and quantitatively.
A diffusion-based architecture unifies garment detail preservation and warping for pose and shape variation in virtual try-on tasks.
diffusion-based architecture Parallel-UNet cross attention mechanism garment detail preservation garment warp person blend TryOnDiffusion