title: SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual
    Editing
summary: Effective editing of personal content holds a pivotal role in enabling
  individuals to express their creativity, weaving captivating narratives within
  their visual stories, and elevate the overall quality and impact of their
  visual content. Therefore, in this work, we introduce SwapAnything, a novel
  framework that can swap any objects in an image with personalized concepts
  given by the reference, while keeping the context unchanged. Compared with
  existing methods for personalized subject swapping, SwapAnything has three
  unique advantages: (1) precise control of arbitrary objects and parts rather
  than the main subject, (2) more faithful preservation of context pixels, (3)
  better adaptation of the personalized concept to the image. First, we propose
  targeted variable swapping to apply region control over latent feature maps and
  swap masked variables for faithful context preservation and initial semantic
  concept swapping. Then, we introduce appearance adaptation, to seamlessly adapt
  the semantic concept into the original image in terms of target location,
  shape, style, and content during the image generation process. Extensive
  results on both human and automatic evaluation demonstrate significant
  improvements of our approach over baseline methods on personalized swapping.
  Furthermore, SwapAnything shows its precise and faithful swapping abilities
  across single object, multiple objects, partial object, and cross-domain
  swapping tasks. SwapAnything also achieves great performance on text-based
  swapping and tasks beyond swapping such as object insertion.
paper_authors._id: 6614c0c084b4d608d5fa518d
paper_authors.name: Jing Gu
paper_authors._id: 6614c0c084b4d608d5fa518e
paper_authors.name: Yilin Wang
paper_authors._id: 6614c0c084b4d608d5fa518f
paper_authors.name: Nanxuan Zhao
paper_authors._id: 6614c0c084b4d608d5fa5190
paper_authors.name: Wei Xiong
paper_authors._id: 6614c0c084b4d608d5fa5191
paper_authors.name: Qing Liu
paper_authors._id: 6614c0c084b4d608d5fa5192
paper_authors.name: Zhifei Zhang
paper_authors._id: 6614c0c084b4d608d5fa5193
paper_authors.name: He Zhang
paper_authors._id: 6614c0c084b4d608d5fa5194
paper_authors.name: Jianming Zhang
paper_authors._id: 6614c0c084b4d608d5fa5195
paper_authors.name: HyunJoon Jung
paper_authors._id: 6614c0c084b4d608d5fa5196
paper_authors.name: Xin Eric Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-09T19:48:20.566Z
paper_authors._id: 64679a226192d39142245e5e
paper_authors.avatarUrl: /avatars/05abee0b6317f100923936ca2099e9eb.svg
paper_authors.fullname: Xin Eric Wang
paper_authors.type: user
paper_authors.user: xw-eric
paper_title: SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual
    Editing
paper_summary: Effective editing of personal content holds a pivotal role in enabling
  individuals to express their creativity, weaving captivating narratives within
  their visual stories, and elevate the overall quality and impact of their
  visual content. Therefore, in this work, we introduce SwapAnything, a novel
  framework that can swap any objects in an image with personalized concepts
  given by the reference, while keeping the context unchanged. Compared with
  existing methods for personalized subject swapping, SwapAnything has three
  unique advantages: (1) precise control of arbitrary objects and parts rather
  than the main subject, (2) more faithful preservation of context pixels, (3)
  better adaptation of the personalized concept to the image. First, we propose
  targeted variable swapping to apply region control over latent feature maps and
  swap masked variables for faithful context preservation and initial semantic
  concept swapping. Then, we introduce appearance adaptation, to seamlessly adapt
  the semantic concept into the original image in terms of target location,
  shape, style, and content during the image generation process. Extensive
  results on both human and automatic evaluation demonstrate significant
  improvements of our approach over baseline methods on personalized swapping.
  Furthermore, SwapAnything shows its precise and faithful swapping abilities
  across single object, multiple objects, partial object, and cross-domain
  swapping tasks. SwapAnything also achieves great performance on text-based
  swapping and tasks beyond swapping such as object insertion.
paper_ai_keywords: targeted variable Obstructing region control latent feature maps faithfulness preservation context pixels appearance adaptation semantic concept swapping image generation process human evaluation automatic evaluation baseline methods precise swapping faithful swapping single object multiple objects partial object cross-domain swapping text-based swapping object insertion