title: Diffuse to Choose: Enriching Image Conditioned Inpainting in Latent
    Diffusion Models for Virtual Try-All
summary: As online shopping is growing, the ability for buyers to virtually visualize
  products in their settings-a phenomenon we define as "Virtual Try-All"-has
  become crucial. Recent diffusion models inherently contain a world model,
  rendering them suitable for this task within an inpainting context. However,
  traditional image-conditioned diffusion models often fail to capture the
  fine-grained details of products. In contrast, personalization-driven models
  such as DreamPaint are good at preserving the item's details but they are not
  optimized for real-time applications. We present "Diffuse to Choose," a novel
  diffusion-based image-conditioned inpainting model that efficiently balances
  fast inference with the retention of high-fidelity details in a given reference
  item while ensuring accurate semantic manipulations in the given scene content.
  Our approach is based on incorporating fine-grained features from the reference
  image directly into the latent feature maps of the main diffusion model,
  alongside with a perceptual loss to further preserve the reference item's
  details. We conduct extensive testing on both in-house and publicly available
  datasets, and show that Diffuse to Choose is superior to existing zero-shot
  diffusion inpainting methods as well as few-shot diffusion personalization
  algorithms like DreamPaint.
paper_authors._id: 65b334dada72b2526d3f9212
paper_authors.name: Mehmet Saygin Seyfioglu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:25:32.062Z
paper_authors._id: 6359e109d72fc0539e767330
paper_authors.avatarUrl: /avatars/a86d541980b56d6532f2bd8078998c74.svg
paper_authors.fullname: Mehmet Saygin Seyfioglu
paper_authors.type: user
paper_authors.user: aldraus
paper_authors._id: 65b334dada72b2526d3f9213
paper_authors.name: Karim Bouyarmane
paper_authors._id: 65b334dada72b2526d3f9214
paper_authors.name: Suren Kumar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:25:46.702Z
paper_authors._id: 657cd73e156fdf03018cfe33
paper_authors.avatarUrl: /avatars/46644cec76c7bca338210beeb3235fb5.svg
paper_authors.fullname: Suren Kumar
paper_authors.type: user
paper_authors.user: SSKumar
paper_authors._id: 65b334dada72b2526d3f9215
paper_authors.name: Amir Tavanaei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:25:54.074Z
paper_authors._id: 638f644bc67af472d31a93a0
paper_authors.avatarUrl: /avatars/b4f2e07ef457789fedcf20a258d8b7b2.svg
paper_authors.fullname: Amir Tavanaei
paper_authors.type: user
paper_authors.user: tavanaei
paper_authors._id: 65b334dada72b2526d3f9216
paper_authors.name: Ismail B. Tutar
paper_title: Diffuse to Choose: Enriching Image Conditioned Inpainting in Latent
    Diffusion Models for Virtual Try-All
paper_summary: As online shopping is growing, the ability for buyers to virtually visualize
  products in their settings-a phenomenon we define as "Virtual Try-All"-has
  become crucial. Recent diffusion models inherently contain a world model,
  rendering them suitable for this task within an inpainting context. However,
  traditional image-conditioned diffusion models often fail to capture the
  fine-grained details of products. In contrast, personalization-driven models
  such as DreamPaint are good at preserving the item's details but they are not
  optimized for real-time applications. We present "Diffuse to Choose," a novel
  diffusion-based image-conditioned inpainting model that efficiently balances
  fast inference with the retention of high-fidelity details in a given reference
  item while ensuring accurate semantic manipulations in the given scene content.
  Our approach is based on incorporating fine-grained features from the reference
  image directly into the latent feature maps of the main diffusion model,
  alongside with a perceptual loss to further preserve the reference item's
  details. We conduct extensive testing on both in-house and publicly available
  datasets, and show that Diffuse to Choose is superior to existing zero-shot
  diffusion inpainting methods as well as few-shot diffusion personalization
  algorithms like DreamPaint.
paper_ai_summary: Diffuse to Choose is a diffusion-based inpainting model that balances real-time performance with high-fidelity detail preservation and semantic accuracy.
paper_ai_keywords: diffusion models image-conditioned diffusion models fine-grained details latent feature maps perceptual loss zero-shot diffusion inpainting few-shot diffusion personalization DreamPaint