DragDiffusion: Harnessing Diffusion Models for Interactive Point-based
  Image Editing
Precise and controllable image editing is a challenging task that has
attracted significant attention. Recently, DragGAN enables an interactive
point-based image editing framework and achieves impressive editing results
with pixel-level precision. However, since this method is based on generative
adversarial networks (GAN), its generality is upper-bounded by the capacity of
the pre-trained GAN models. In this work, we extend such an editing framework
to diffusion models and propose DragDiffusion. By leveraging large-scale
pretrained diffusion models, we greatly improve the applicability of
interactive point-based editing in real world scenarios. While most existing
diffusion-based image editing methods work on text embeddings, DragDiffusion
optimizes the diffusion latent to achieve precise spatial control. Although
diffusion models generate images in an iterative manner, we empirically show
that optimizing diffusion latent at one single step suffices to generate
coherent results, enabling DragDiffusion to complete high-quality editing
efficiently. Extensive experiments across a wide range of challenging cases
(e.g., multi-objects, diverse object categories, various styles, etc.)
demonstrate the versatility and generality of DragDiffusion.
649a792ea37f2e73185165e1
Yujun Shi
claimed_verified
2023-06-27T12:55:23.745Z
645ce309a03f3ebb0bdf2b89
/avatars/652db4d2badd1191c6612f0950a88a88.svg
Yujun Shi
user
PeiyangShi
649a792ea37f2e73185165e2
Chuhui Xue
649a792ea37f2e73185165e3
Jiachun Pan
649a792ea37f2e73185165e4
Wenqing Zhang
649a792ea37f2e73185165e5
Vincent Y. F. Tan
649a792ea37f2e73185165e6
Song Bai
DragDiffusion: Harnessing Diffusion Models for Interactive Point-based
  Image Editing
Precise and controllable image editing is a challenging task that has
attracted significant attention. Recently, DragGAN enables an interactive
point-based image editing framework and achieves impressive editing results
with pixel-level precision. However, since this method is based on generative
adversarial networks (GAN), its generality is upper-bounded by the capacity of
the pre-trained GAN models. In this work, we extend such an editing framework
to diffusion models and propose DragDiffusion. By leveraging large-scale
pretrained diffusion models, we greatly improve the applicability of
interactive point-based editing in real world scenarios. While most existing
diffusion-based image editing methods work on text embeddings, DragDiffusion
optimizes the diffusion latent to achieve precise spatial control. Although
diffusion models generate images in an iterative manner, we empirically show
that optimizing diffusion latent at one single step suffices to generate
coherent results, enabling DragDiffusion to complete high-quality editing
efficiently. Extensive experiments across a wide range of challenging cases
(e.g., multi-objects, diverse object categories, various styles, etc.)
demonstrate the versatility and generality of DragDiffusion.
DragDiffusion extends interactive point-based image editing to diffusion models, offering precise spatial control and high-quality editing across diverse scenarios.
generative adversarial networks diffusion models diffusion latent iterative manner interactive point-based editing multi-objects diverse object categories various styles