title: DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models
summary: Despite the ability of existing large-scale text-to-image (T2I) models to
  generate high-quality images from detailed textual descriptions, they often
  lack the ability to precisely edit the generated or real images. In this paper,
  we propose a novel image editing method, DragonDiffusion, enabling Drag-style
  manipulation on Diffusion models. Specifically, we construct classifier
  guidance based on the strong correspondence of intermediate features in the
  diffusion model. It can transform the editing signals into gradients via
  feature correspondence loss to modify the intermediate representation of the
  diffusion model. Based on this guidance strategy, we also build a multi-scale
  guidance to consider both semantic and geometric alignment. Moreover, a
  cross-branch self-attention is added to maintain the consistency between the
  original image and the editing result. Our method, through an efficient design,
  achieves various editing modes for the generated or real images, such as object
  moving, object resizing, object appearance replacement, and content dragging.
  It is worth noting that all editing and content preservation signals come from
  the image itself, and the model does not require fine-tuning or additional
  modules. Our source code will be available at
  https://github.com/MC-E/DragonDiffusion.
paper_authors._id: 64a641220c2ce52c64ee6b68
paper_authors.name: Chong Mou
paper_authors._id: 64a641220c2ce52c64ee6b69
paper_authors.name: Xintao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-07T07:35:04.685Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 64a641220c2ce52c64ee6b6a
paper_authors.name: Jiechong Song
paper_authors._id: 64a641220c2ce52c64ee6b6b
paper_authors.name: Ying Shan
paper_authors._id: 64a641220c2ce52c64ee6b6c
paper_authors.name: Jian Zhang
paper_title: DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models
paper_summary: Despite the ability of existing large-scale text-to-image (T2I) models to
  generate high-quality images from detailed textual descriptions, they often
  lack the ability to precisely edit the generated or real images. In this paper,
  we propose a novel image editing method, DragonDiffusion, enabling Drag-style
  manipulation on Diffusion models. Specifically, we construct classifier
  guidance based on the strong correspondence of intermediate features in the
  diffusion model. It can transform the editing signals into gradients via
  feature correspondence loss to modify the intermediate representation of the
  diffusion model. Based on this guidance strategy, we also build a multi-scale
  guidance to consider both semantic and geometric alignment. Moreover, a
  cross-branch self-attention is added to maintain the consistency between the
  original image and the editing result. Our method, through an efficient design,
  achieves various editing modes for the generated or real images, such as object
  moving, object resizing, object appearance replacement, and content dragging.
  It is worth noting that all editing and content preservation signals come from
  the image itself, and the model does not require fine-tuning or additional
  modules. Our source code will be available at
  https://github.com/MC-E/DragonDiffusion.
paper_ai_keywords: text-to-image (T2I) models DragonDiffusion classifier guidance diffusion model feature correspondence loss intermediate features multi-scale guidance cross-branch self-attention Drag-style manipulation object moving object resizing object appearance replacement content dragging