title: UltraEdit: Instruction-based Fine-Grained Image Editing at Scale
summary: This paper presents UltraEdit, a large-scale (approximately 4 million editing
  samples), automatically generated dataset for instruction-based image editing.
  Our key idea is to address the drawbacks in existing image editing datasets
  like InstructPix2Pix and MagicBrush, and provide a systematic approach to
  producing massive and high-quality image editing samples. UltraEdit offers
  several distinct advantages: 1) It features a broader range of editing
  instructions by leveraging the creativity of large language models (LLMs)
  alongside in-context editing examples from human raters; 2) Its data sources
  are based on real images, including photographs and artworks, which provide
  greater diversity and reduced bias compared to datasets solely generated by
  text-to-image models; 3) It also supports region-based editing, enhanced by
  high-quality, automatically produced region annotations. Our experiments show
  that canonical diffusion-based editing baselines trained on UltraEdit set new
  records on MagicBrush and Emu-Edit benchmarks. Our analysis further confirms
  the crucial role of real image anchors and region-based editing data. The
  dataset, code, and models can be found in https://ultra-editing.github.io.
paper_authors._id: 668d5fd3ed63008dfaad9325
paper_authors.name: Haozhe Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-22T08:43:35.693Z
paper_authors._id: 63ff09a02098b9ad105a09f6
paper_authors.avatarUrl: /avatars/4409ca5d320050cf4c3df05962c7ff58.svg
paper_authors.fullname: Hans Zhao
paper_authors.type: user
paper_authors.user: BleachNick
paper_authors._id: 668d5fd3ed63008dfaad9326
paper_authors.name: Xiaojian Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T11:59:23.292Z
paper_authors._id: 60dd0e36a15ddd7d2006d2e9
paper_authors.avatarUrl: /avatars/8bd98177a79efbf295be8f6457683297.svg
paper_authors.fullname: Xiaojian Ma
paper_authors.type: user
paper_authors.user: jeasinema
paper_authors._id: 668d5fd3ed63008dfaad9327
paper_authors.name: Liang Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T11:59:31.061Z
paper_authors._id: 658c481dd1c8b106727a8b73
paper_authors.avatarUrl: /avatars/d34a7a62c3a524e5fdd2d5994348db58.svg
paper_authors.fullname: Liang Chen
paper_authors.type: user
paper_authors.user: liangchen-ms
paper_authors._id: 668d5fd3ed63008dfaad9328
paper_authors.name: Shuzheng Si
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T07:45:34.954Z
paper_authors._id: 637c99bbfe115289cfedfb44
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/637c99bbfe115289cfedfb44/344NN9KKF_XXTlVYaGaMW.png
paper_authors.fullname: ssz
paper_authors.type: user
paper_authors.user: ssz1111
paper_authors._id: 668d5fd3ed63008dfaad9329
paper_authors.name: Rujie Wu
paper_authors._id: 668d5fd3ed63008dfaad932a
paper_authors.name: Kaikai An
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-23T07:35:41.139Z
paper_authors._id: 66c89152d33e34fbc29497d7
paper_authors.avatarUrl: /avatars/bbddabf6532393951c4759e5915a065b.svg
paper_authors.fullname: KaikaiAn
paper_authors.type: user
paper_authors.user: kkk-an
paper_authors._id: 668d5fd3ed63008dfaad932b
paper_authors.name: Peiyu Yu
paper_authors._id: 668d5fd3ed63008dfaad932c
paper_authors.name: Minjia Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T12:00:06.212Z
paper_authors._id: 6305b5e39d2531fabd195c5f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6305b5e39d2531fabd195c5f/oGS5leXVrB8Ct_BF96VkR.jpeg
paper_authors.fullname: Zhang
paper_authors.type: user
paper_authors.user: Minjia
paper_authors._id: 668d5fd3ed63008dfaad932d
paper_authors.name: Qing Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-03T08:26:10.560Z
paper_authors._id: 6455f8c2c8f569b995d603c9
paper_authors.avatarUrl: /avatars/3cb2cad3ab123887c270234bb6a4ca43.svg
paper_authors.fullname: Qing Li
paper_authors.type: user
paper_authors.user: li-qing
paper_authors._id: 668d5fd3ed63008dfaad932e
paper_authors.name: Baobao Chang
paper_title: UltraEdit: Instruction-based Fine-Grained Image Editing at Scale
paper_summary: This paper presents UltraEdit, a large-scale (approximately 4 million editing
  samples), automatically generated dataset for instruction-based image editing.
  Our key idea is to address the drawbacks in existing image editing datasets
  like InstructPix2Pix and MagicBrush, and provide a systematic approach to
  producing massive and high-quality image editing samples. UltraEdit offers
  several distinct advantages: 1) It features a broader range of editing
  instructions by leveraging the creativity of large language models (LLMs)
  alongside in-context editing examples from human raters; 2) Its data sources
  are based on real images, including photographs and artworks, which provide
  greater diversity and reduced bias compared to datasets solely generated by
  text-to-image models; 3) It also supports region-based editing, enhanced by
  high-quality, automatically produced region annotations. Our experiments show
  that canonical diffusion-based editing baselines trained on UltraEdit set new
  records on MagicBrush and Emu-Edit benchmarks. Our analysis further confirms
  the crucial role of real image anchors and region-based editing data. The
  dataset, code, and models can be found in https://ultra-editing.github.io.
paper_ai_keywords: BigEdit UltraEdit InstructPix2Pix MagicBrush large-scale dataset instruction-based image editing image editing dataset automatic generation large language models (LLMs) hallucinations editing examples real images photographs artworks diversity bias text-to-image models region-based editing region annotations canonical diffusion-based editing UltraEdit benchmarks Emu-Edit benchmarks real image anchors