title: DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image
    Editing
summary: Large-scale Text-to-Image (T2I) diffusion models have revolutionized image
  generation over the last few years. Although owning diverse and high-quality
  generation capabilities, translating these abilities to fine-grained image
  editing remains challenging. In this paper, we propose DiffEditor to rectify
  two weaknesses in existing diffusion-based image editing: (1) in complex
  scenarios, editing results often lack editing accuracy and exhibit unexpected
  artifacts; (2) lack of flexibility to harmonize editing operations, e.g.,
  imagine new content. In our solution, we introduce image prompts in
  fine-grained image editing, cooperating with the text prompt to better describe
  the editing content. To increase the flexibility while maintaining content
  consistency, we locally combine stochastic differential equation (SDE) into the
  ordinary differential equation (ODE) sampling. In addition, we incorporate
  regional score-based gradient guidance and a time travel strategy into the
  diffusion sampling, further improving the editing quality. Extensive
  experiments demonstrate that our method can efficiently achieve
  state-of-the-art performance on various fine-grained image editing tasks,
  including editing within a single image (e.g., object moving, resizing, and
  content dragging) and across images (e.g., appearance replacing and object
  pasting). Our source code is released at
  https://github.com/MC-E/DragonDiffusion.
paper_authors._id: 65c1a36355c4f06fa96aa3fb
paper_authors.name: Chong Mou
paper_authors._id: 65c1a36355c4f06fa96aa3fc
paper_authors.name: Xintao Wang
paper_authors._id: 65c1a36355c4f06fa96aa3fd
paper_authors.name: Jiechong Song
paper_authors._id: 65c1a36355c4f06fa96aa3fe
paper_authors.name: Ying Shan
paper_authors._id: 65c1a36355c4f06fa96aa3ff
paper_authors.name: Jian Zhang
paper_title: DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image
    Editing
paper_summary: Large-scale Text-to-Image (T2I) diffusion models have revolutionized image
  generation over the last few years. Although owning diverse and high-quality
  generation capabilities, translating these abilities to fine-grained image
  editing remains challenging. In this paper, we propose DiffEditor to rectify
  two weaknesses in existing diffusion-based image editing: (1) in complex
  scenarios, editing results often lack editing accuracy and exhibit unexpected
  artifacts; (2) lack of flexibility to harmonize editing operations, e.g.,
  imagine new content. In our solution, we introduce image prompts in
  fine-grained image editing, cooperating with the text prompt to better describe
  the editing content. To increase the flexibility while maintaining content
  consistency, we locally combine stochastic differential equation (SDE) into the
  ordinary differential equation (ODE) sampling. In addition, we incorporate
  regional score-based gradient guidance and a time travel strategy into the
  diffusion sampling, further improving the editing quality. Extensive
  experiments demonstrate that our method can efficiently achieve
  state-of-the-art performance on various fine-grained image editing tasks,
  including editing within a single image (e.g., object moving, resizing, and
  content dragging) and across images (e.g., appearance replacing and object
  pasting). Our source code is released at
  https://github.com/MC-E/DragonDiffusion.
paper_ai_keywords: diffusion models fine-grained image editing image prompts text prompt stochastic differential equation (SDE) ordinary differential equation (ODE) sampling regional score-based gradient guidance time travel strategy diffusion sampling object moving resizing content dragging appearance replacing object pasting