title: DragDiffusion: Harnessing Diffusion Models for Interactive Point-based
    Image Editing
summary: Precise and controllable image editing is a challenging task that has
  attracted significant attention. Recently, DragGAN enables an interactive
  point-based image editing framework and achieves impressive editing results
  with pixel-level precision. However, since this method is based on generative
  adversarial networks (GAN), its generality is upper-bounded by the capacity of
  the pre-trained GAN models. In this work, we extend such an editing framework
  to diffusion models and propose DragDiffusion. By leveraging large-scale
  pretrained diffusion models, we greatly improve the applicability of
  interactive point-based editing in real world scenarios. While most existing
  diffusion-based image editing methods work on text embeddings, DragDiffusion
  optimizes the diffusion latent to achieve precise spatial control. Although
  diffusion models generate images in an iterative manner, we empirically show
  that optimizing diffusion latent at one single step suffices to generate
  coherent results, enabling DragDiffusion to complete high-quality editing
  efficiently. Extensive experiments across a wide range of challenging cases
  (e.g., multi-objects, diverse object categories, various styles, etc.)
  demonstrate the versatility and generality of DragDiffusion.
paper_authors._id: 649a792ea37f2e73185165e1
paper_authors.name: Yujun Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-27T12:55:23.745Z
paper_authors._id: 645ce309a03f3ebb0bdf2b89
paper_authors.avatarUrl: /avatars/652db4d2badd1191c6612f0950a88a88.svg
paper_authors.fullname: Yujun Shi
paper_authors.type: user
paper_authors.user: PeiyangShi
paper_authors._id: 649a792ea37f2e73185165e2
paper_authors.name: Chuhui Xue
paper_authors._id: 649a792ea37f2e73185165e3
paper_authors.name: Jiachun Pan
paper_authors._id: 649a792ea37f2e73185165e4
paper_authors.name: Wenqing Zhang
paper_authors._id: 649a792ea37f2e73185165e5
paper_authors.name: Vincent Y. F. Tan
paper_authors._id: 649a792ea37f2e73185165e6
paper_authors.name: Song Bai
paper_title: DragDiffusion: Harnessing Diffusion Models for Interactive Point-based
    Image Editing
paper_summary: Precise and controllable image editing is a challenging task that has
  attracted significant attention. Recently, DragGAN enables an interactive
  point-based image editing framework and achieves impressive editing results
  with pixel-level precision. However, since this method is based on generative
  adversarial networks (GAN), its generality is upper-bounded by the capacity of
  the pre-trained GAN models. In this work, we extend such an editing framework
  to diffusion models and propose DragDiffusion. By leveraging large-scale
  pretrained diffusion models, we greatly improve the applicability of
  interactive point-based editing in real world scenarios. While most existing
  diffusion-based image editing methods work on text embeddings, DragDiffusion
  optimizes the diffusion latent to achieve precise spatial control. Although
  diffusion models generate images in an iterative manner, we empirically show
  that optimizing diffusion latent at one single step suffices to generate
  coherent results, enabling DragDiffusion to complete high-quality editing
  efficiently. Extensive experiments across a wide range of challenging cases
  (e.g., multi-objects, diverse object categories, various styles, etc.)
  demonstrate the versatility and generality of DragDiffusion.
paper_ai_summary: DragDiffusion extends interactive point-based image editing to diffusion models, offering precise spatial control and high-quality editing across diverse scenarios.
paper_ai_keywords: generative adversarial networks diffusion models diffusion latent iterative manner interactive point-based editing multi-objects diverse object categories various styles