title: MAG-Edit: Localized Image Editing in Complex Scenarios via
    Mask-Based Attention-Adjusted
    Guidance
summary: Recent diffusion-based image editing approaches have exhibited impressive
  editing capabilities in images with simple compositions. However, localized
  editing in complex scenarios has not been well-studied in the literature,
  despite its growing real-world demands. Existing mask-based inpainting methods
  fall short of retaining the underlying structure within the edit region.
  Meanwhile, mask-free attention-based methods often exhibit editing leakage and
  misalignment in more complex compositions. In this work, we develop
  MAG-Edit, a training-free, inference-stage optimization method,
  which enables localized image editing in complex scenarios. In particular,
  MAG-Edit optimizes the noise latent feature in diffusion models by maximizing
  two mask-based cross-attention constraints of the edit token, which in turn
  gradually enhances the local alignment with the desired prompt. Extensive
  quantitative and qualitative experiments demonstrate the effectiveness of our
  method in achieving both text alignment and structure preservation for
  localized editing within complex scenarios.
paper_authors._id: 65813b40560c1557cfdc9690
paper_authors.name: Qi Mao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-19T10:16:53.496Z
paper_authors._id: 6388a7e98a5dbe2f3dc61faa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6388a7e98a5dbe2f3dc61faa/Zj8pR7BG_wOQjsYzosb5p.jpeg
paper_authors.fullname: Qi Mao
paper_authors.type: user
paper_authors.user: HelenMao
paper_authors._id: 65813b40560c1557cfdc9691
paper_authors.name: Lan Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-19T10:53:47.421Z
paper_authors._id: 640d704c8036cc2142299c19
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640d704c8036cc2142299c19/Wt9AslcVxWOSSc11epk8l.jpeg
paper_authors.fullname: Lan Chen
paper_authors.type: user
paper_authors.user: Orannue
paper_authors._id: 65813b40560c1557cfdc9692
paper_authors.name: Yuchao Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T09:51:21.943Z
paper_authors._id: 63021630a35b21bd8a53305a
paper_authors.avatarUrl: /avatars/7a7e8b39749eda61e57d8a1908726558.svg
paper_authors.fullname: Gu Yuchao
paper_authors.type: user
paper_authors.user: guyuchao
paper_authors._id: 65813b40560c1557cfdc9693
paper_authors.name: Zhen Fang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-19T10:53:55.026Z
paper_authors._id: 64b0a5037a475fba70a7260d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b0a5037a475fba70a7260d/MauBbb6raMA23yrR1Zq21.jpeg
paper_authors.fullname: Zhen Fang
paper_authors.type: user
paper_authors.user: CostaliyA
paper_authors._id: 65813b40560c1557cfdc9694
paper_authors.name: Mike Zheng Shou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T09:50:47.842Z
paper_authors._id: 63a55320ce5763e06f78519c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
paper_authors.fullname: Mike Shou
paper_authors.type: user
paper_authors.user: mikeshou
paper_title: MAG-Edit: Localized Image Editing in Complex Scenarios via
    Mask-Based Attention-Adjusted
    Guidance
paper_summary: Recent diffusion-based image editing approaches have exhibited impressive
  editing capabilities in images with simple compositions. However, localized
  editing in complex scenarios has not been well-studied in the literature,
  despite its growing real-world demands. Existing mask-based inpainting methods
  fall short of retaining the underlying structure within the edit region.
  Meanwhile, mask-free attention-based methods often exhibit editing leakage and
  misalignment in more complex compositions. In this work, we develop
  MAG-Edit, a training-free, inference-stage optimization method,
  which enables localized image editing in complex scenarios. In particular,
  MAG-Edit optimizes the noise latent feature in diffusion models by maximizing
  two mask-based cross-attention constraints of the edit token, which in turn
  gradually enhances the local alignment with the desired prompt. Extensive
  quantitative and qualitative experiments demonstrate the effectiveness of our
  method in achieving both text alignment and structure preservation for
  localized editing within complex scenarios.
paper_ai_summary: MAG-Edit enhances localized image editing in complex scenarios by optimizing noise latent features in diffusion models through mask-based cross-attention constraints.
paper_ai_keywords: diffusion models noise latent feature cross-attention edit token text alignment structure preservation