title: MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image
    Editing
summary: Text-guided image editing is widely needed in daily life, ranging from
  personal use to professional applications such as Photoshop. However, existing
  methods are either zero-shot or trained on an automatically synthesized
  dataset, which contains a high volume of noise. Thus, they still require lots
  of manual tuning to produce desirable outcomes in practice. To address this
  issue, we introduce MagicBrush (https://osu-nlp-group.github.io/MagicBrush/),
  the first large-scale, manually annotated dataset for instruction-guided real
  image editing that covers diverse scenarios: single-turn, multi-turn,
  mask-provided, and mask-free editing. MagicBrush comprises over 10K manually
  annotated triples (source image, instruction, target image), which supports
  trainining large-scale text-guided image editing models. We fine-tune
  InstructPix2Pix on MagicBrush and show that the new model can produce much
  better images according to human evaluation. We further conduct extensive
  experiments to evaluate current image editing baselines from multiple
  dimensions including quantitative, qualitative, and human evaluations. The
  results reveal the challenging nature of our dataset and the gap between
  current baselines and real-world editing needs.
paper_authors._id: 648faa9519234cd09c480d22
paper_authors.name: Kai Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-03T07:28:03.726Z
paper_authors._id: 63e0a50242591dda0b9dca5c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63e0a50242591dda0b9dca5c/c7cBPEBWQDFYimfGnO_SI.png
paper_authors.fullname: Kai Zhang
paper_authors.type: user
paper_authors.user: drogozhang
paper_authors._id: 648faa9519234cd09c480d23
paper_authors.name: Lingbo Mo
paper_authors._id: 648faa9519234cd09c480d24
paper_authors.name: Wenhu Chen
paper_authors._id: 648faa9519234cd09c480d25
paper_authors.name: Huan Sun
paper_authors._id: 648faa9519234cd09c480d26
paper_authors.name: Yu Su
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-20T08:50:10.908Z
paper_authors._id: 6477a323dbc2a416f8b852b3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6477a323dbc2a416f8b852b3/mRKW5kT9GASORT4YnaZz0.jpeg
paper_authors.fullname: Yu Su
paper_authors.type: user
paper_authors.user: ysu-nlp
paper_title: MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image
    Editing
paper_summary: Text-guided image editing is widely needed in daily life, ranging from
  personal use to professional applications such as Photoshop. However, existing
  methods are either zero-shot or trained on an automatically synthesized
  dataset, which contains a high volume of noise. Thus, they still require lots
  of manual tuning to produce desirable outcomes in practice. To address this
  issue, we introduce MagicBrush (https://osu-nlp-group.github.io/MagicBrush/),
  the first large-scale, manually annotated dataset for instruction-guided real
  image editing that covers diverse scenarios: single-turn, multi-turn,
  mask-provided, and mask-free editing. MagicBrush comprises over 10K manually
  annotated triples (source image, instruction, target image), which supports
  trainining large-scale text-guided image editing models. We fine-tune
  InstructPix2Pix on MagicBrush and show that the new model can produce much
  better images according to human evaluation. We further conduct extensive
  experiments to evaluate current image editing baselines from multiple
  dimensions including quantitative, qualitative, and human evaluations. The
  results reveal the challenging nature of our dataset and the gap between
  current baselines and real-world editing needs.
paper_ai_summary: MagicBrush, a large-scale manually annotated dataset for text-guided real image editing, enhances InstructPix2Pix model performance and highlights challenges in current baselines.
paper_ai_keywords: InstructPix2Pix text-guided image editing manually annotated source image instruction target image human evaluation