TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction
  using Diffusion Models
This paper introduces Virtual Try-Off (VTOFF), a novel task focused on
generating standardized garment images from single photos of clothed
individuals. Unlike traditional Virtual Try-On (VTON), which digitally dresses
models, VTOFF aims to extract a canonical garment image, posing unique
challenges in capturing garment shape, texture, and intricate patterns. This
well-defined target makes VTOFF particularly effective for evaluating
reconstruction fidelity in generative models. We present TryOffDiff, a model
that adapts Stable Diffusion with SigLIP-based visual conditioning to ensure
high fidelity and detail retention. Experiments on a modified VITON-HD dataset
show that our approach outperforms baseline methods based on pose transfer and
virtual try-on with fewer pre- and post-processing steps. Our analysis reveals
that traditional image generation metrics inadequately assess reconstruction
quality, prompting us to rely on DISTS for more accurate evaluation. Our
results highlight the potential of VTOFF to enhance product imagery in
e-commerce applications, advance generative model evaluation, and inspire
future work on high-fidelity reconstruction. Demo, code, and models are
available at: https://rizavelioglu.github.io/tryoffdiff/
674818fd93aea005d45001bd
Riza Velioglu
claimed_verified
2024-11-28T09:19:21.403Z
63f222ecf4e30ffd2bd08405
https://cdn-avatars.huggingface.co/v1/production/uploads/63f222ecf4e30ffd2bd08405/AJoV9-gbiwKcOFPIGPDvD.jpeg
Riza Velioglu
user
rizavelioglu
674818fd93aea005d45001be
Petra Bevandic
admin_assigned
2024-11-29T15:25:20.925Z
66c7334bf486e7ed57231858
/avatars/2f315177d64b683b32d7d4ef6a6c91eb.svg
Petra Bevandic
user
PetraBevandic
674818fd93aea005d45001bf
Robin Chan
claimed_verified
2024-12-02T09:46:16.168Z
64491f1bb6ac93fe65149437
/avatars/8553dd8a9e4972b90069b57998d8d8ca.svg
Robin Chan
user
robin-chan
674818fd93aea005d45001c0
Barbara Hammer
TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction
  using Diffusion Models
This paper introduces Virtual Try-Off (VTOFF), a novel task focused on
generating standardized garment images from single photos of clothed
individuals. Unlike traditional Virtual Try-On (VTON), which digitally dresses
models, VTOFF aims to extract a canonical garment image, posing unique
challenges in capturing garment shape, texture, and intricate patterns. This
well-defined target makes VTOFF particularly effective for evaluating
reconstruction fidelity in generative models. We present TryOffDiff, a model
that adapts Stable Diffusion with SigLIP-based visual conditioning to ensure
high fidelity and detail retention. Experiments on a modified VITON-HD dataset
show that our approach outperforms baseline methods based on pose transfer and
virtual try-on with fewer pre- and post-processing steps. Our analysis reveals
that traditional image generation metrics inadequately assess reconstruction
quality, prompting us to rely on DISTS for more accurate evaluation. Our
results highlight the potential of VTOFF to enhance product imagery in
e-commerce applications, advance generative model evaluation, and inspire
future work on high-fidelity reconstruction. Demo, code, and models are
available at: https://rizavelioglu.github.io/tryoffdiff/
TryOffDiff, an adaptation of Stable Diffusion for generating standardized garment images from dressed individuals, outperforms pose transfer and virtual try-on methods using SigLIP-based visual conditioning and the DISTS metric for evaluation.
Virtual Try-Off Stable Diffusion SigLIP-based visual conditioning DISTS