title: Magic Insert: Style-Aware Drag-and-Drop
summary: We present Magic Insert, a method for dragging-and-dropping subjects from a
  user-provided image into a target image of a different style in a physically
  plausible manner while matching the style of the target image. This work
  formalizes the problem of style-aware drag-and-drop and presents a method for
  tackling it by addressing two sub-problems: style-aware personalization and
  realistic object insertion in stylized images. For style-aware personalization,
  our method first fine-tunes a pretrained text-to-image diffusion model using
  LoRA and learned text tokens on the subject image, and then infuses it with a
  CLIP representation of the target style. For object insertion, we use
  Bootstrapped Domain Adaption to adapt a domain-specific photorealistic object
  insertion model to the domain of diverse artistic styles. Overall, the method
  significantly outperforms traditional approaches such as inpainting. Finally,
  we present a dataset, SubjectPlop, to facilitate evaluation and future progress
  in this area. Project page: https://magicinsert.github.io/
paper_authors._id: 66857aa5ef8d151431928941
paper_authors.name: Nataniel Ruiz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-04T08:03:58.411Z
paper_authors._id: 6318c6e6cb116eab31d70a16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6318c6e6cb116eab31d70a16/yAw5XtvqaPUkPj2B6JOzz.jpeg
paper_authors.fullname: Nataniel
paper_authors.type: user
paper_authors.user: flavoredquark
paper_authors._id: 66857aa5ef8d151431928942
paper_authors.name: Yuanzhen Li
paper_authors._id: 66857aa5ef8d151431928943
paper_authors.name: Neal Wadhwa
paper_authors._id: 66857aa5ef8d151431928944
paper_authors.name: Yael Pritch
paper_authors._id: 66857aa5ef8d151431928945
paper_authors.name: Michael Rubinstein
paper_authors._id: 66857aa5ef8d151431928946
paper_authors.name: David E. Jacobs
paper_authors._id: 66857aa5ef8d151431928947
paper_authors.name: Shlomi Fruchter
paper_title: Magic Insert: Style-Aware Drag-and-Drop
paper_summary: We present Magic Insert, a method for dragging-and-dropping subjects from a
  user-provided image into a target image of a different style in a physically
  plausible manner while matching the style of the target image. This work
  formalizes the problem of style-aware drag-and-drop and presents a method for
  tackling it by addressing two sub-problems: style-aware personalization and
  realistic object insertion in stylized images. For style-aware personalization,
  our method first fine-tunes a pretrained text-to-image diffusion model using
  LoRA and learned text tokens on the subject image, and then infuses it with a
  CLIP representation of the target style. For object insertion, we use
  Bootstrapped Domain Adaption to adapt a domain-specific photorealistic object
  insertion model to the domain of diverse artistic styles. Overall, the method
  significantly outperforms traditional approaches such as inpainting. Finally,
  we present a dataset, SubjectPlop, to facilitate evaluation and future progress
  in this area. Project page: https://magicinsert.github.io/
paper_ai_summary: Magic Insert allows style-aware drag-and-drop insertion of subjects from one image into another of a different style using domain adaptation and a fine-tuned diffusion model.
paper_ai_keywords: text-to-image diffusion model LoRA text tokens CLIP representation Bootstrapped Domain Adaptation photorealistic object insertion model style-aware drag-and-drop style-aware personalization realistic object insertion stylized images inpainting