title: TryOnDiffusion: A Tale of Two UNets
summary: Given two images depicting a person and a garment worn by another person, our
  goal is to generate a visualization of how the garment might look on the input
  person. A key challenge is to synthesize a photorealistic detail-preserving
  visualization of the garment, while warping the garment to accommodate a
  significant body pose and shape change across the subjects. Previous methods
  either focus on garment detail preservation without effective pose and shape
  variation, or allow try-on with the desired shape and pose but lack garment
  details. In this paper, we propose a diffusion-based architecture that unifies
  two UNets (referred to as Parallel-UNet), which allows us to preserve garment
  details and warp the garment for significant pose and body change in a single
  network. The key ideas behind Parallel-UNet include: 1) garment is warped
  implicitly via a cross attention mechanism, 2) garment warp and person blend
  happen as part of a unified process as opposed to a sequence of two separate
  tasks. Experimental results indicate that TryOnDiffusion achieves
  state-of-the-art performance both qualitatively and quantitatively.
paper_authors._id: 648bf365f11318e0a448a91c
paper_authors.name: Luyang Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-16T07:06:44.785Z
paper_authors._id: 6426836b13a9e5d9675a6571
paper_authors.avatarUrl: /avatars/6b2925901e36fddb95760e3cbd20d8d7.svg
paper_authors.fullname: Luyang Zhu
paper_authors.type: user
paper_authors.user: lyzhu
paper_authors._id: 648bf365f11318e0a448a91d
paper_authors.name: Dawei Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-16T07:07:01.097Z
paper_authors._id: 6480e70d8c6a3b8f11fae86f
paper_authors.avatarUrl: /avatars/aab6be8626d9a2a87f58c8a5803c72f0.svg
paper_authors.fullname: DaweiYang
paper_authors.type: user
paper_authors.user: DaweiYang
paper_authors._id: 648bf365f11318e0a448a91e
paper_authors.name: Tyler Zhu
paper_authors._id: 648bf365f11318e0a448a91f
paper_authors.name: Fitsum Reda
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-03T14:26:27.153Z
paper_authors._id: 672170c49dd8f8814e66a245
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/672170c49dd8f8814e66a245/Ksfb4iJ5AvlF5NAekH5Ww.jpeg
paper_authors.fullname: Fitsum Reda
paper_authors.type: user
paper_authors.user: fitsumreda
paper_authors._id: 648bf365f11318e0a448a920
paper_authors.name: William Chan
paper_authors._id: 648bf365f11318e0a448a921
paper_authors.name: Chitwan Saharia
paper_authors._id: 648bf365f11318e0a448a922
paper_authors.name: Mohammad Norouzi
paper_authors._id: 648bf365f11318e0a448a923
paper_authors.name: Ira Kemelmacher-Shlizerman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-02T07:20:19.754Z
paper_authors._id: 66d20e411ba71ac4c0488132
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/kkqDAFT15lx4wrJVXiFGJ.png
paper_authors.fullname: Ira Kemelmacher-Shlizerman
paper_authors.type: user
paper_authors.user: kemelmi
paper_title: TryOnDiffusion: A Tale of Two UNets
paper_summary: Given two images depicting a person and a garment worn by another person, our
  goal is to generate a visualization of how the garment might look on the input
  person. A key challenge is to synthesize a photorealistic detail-preserving
  visualization of the garment, while warping the garment to accommodate a
  significant body pose and shape change across the subjects. Previous methods
  either focus on garment detail preservation without effective pose and shape
  variation, or allow try-on with the desired shape and pose but lack garment
  details. In this paper, we propose a diffusion-based architecture that unifies
  two UNets (referred to as Parallel-UNet), which allows us to preserve garment
  details and warp the garment for significant pose and body change in a single
  network. The key ideas behind Parallel-UNet include: 1) garment is warped
  implicitly via a cross attention mechanism, 2) garment warp and person blend
  happen as part of a unified process as opposed to a sequence of two separate
  tasks. Experimental results indicate that TryOnDiffusion achieves
  state-of-the-art performance both qualitatively and quantitatively.
paper_ai_summary: A diffusion-based architecture unifies garment detail preservation and warping for pose and shape variation in virtual try-on tasks.
paper_ai_keywords: diffusion-based architecture Parallel-UNet cross attention mechanism garment detail preservation garment warp person blend TryOnDiffusion