title: OmniEdit: Building Image Editing Generalist Models Through Specialist
    Supervision
summary: Instruction-guided image editing methods have demonstrated significant
  potential by training diffusion models on automatically synthesized or manually
  annotated image editing pairs. However, these methods remain far from
  practical, real-life applications. We identify three primary challenges
  contributing to this gap. Firstly, existing models have limited editing skills
  due to the biased synthesis process. Secondly, these methods are trained with
  datasets with a high volume of noise and artifacts. This is due to the
  application of simple filtering methods like CLIP-score. Thirdly, all these
  datasets are restricted to a single low resolution and fixed aspect ratio,
  limiting the versatility to handle real-world use cases. In this paper, we
  present \omniedit, which is an omnipotent editor to handle seven different
  image editing tasks with any aspect ratio seamlessly. Our contribution is in
  four folds: (1) \omniedit is trained by utilizing the supervision from seven
  different specialist models to ensure task coverage. (2) we utilize importance
  sampling based on the scores provided by large multimodal models (like GPT-4o)
  instead of CLIP-score to improve the data quality. (3) we propose a new editing
  architecture called EditNet to greatly boost the editing success rate, (4) we
  provide images with different aspect ratios to ensure that our model can handle
  any image in the wild. We have curated a test set containing images of
  different aspect ratios, accompanied by diverse instructions to cover different
  tasks. Both automatic evaluation and human evaluations demonstrate that
  \omniedit can significantly outperform all the existing models. Our code,
  dataset and model will be available at
  https://tiger-ai-lab.github.io/OmniEdit/
paper_authors._id: 6732d435f014ee495567a302
paper_authors.name: Cong Wei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:26:52.427Z
paper_authors._id: 64f8e358766ff9f3d2b0de84
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64f8e358766ff9f3d2b0de84/R2P1YG-mRBh7TU9wkjGGk.jpeg
paper_authors.fullname: Cong Wei
paper_authors.type: user
paper_authors.user: CongWei1230
paper_authors._id: 6732d435f014ee495567a303
paper_authors.name: Zheyang Xiong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:27:00.558Z
paper_authors._id: 67084dbb652deb3c09d8f633
paper_authors.avatarUrl: /avatars/6489cc414ec6b7be3523e335ba14888f.svg
paper_authors.fullname: Zheyang Xiong
paper_authors.type: user
paper_authors.user: zheyang-hf
paper_authors._id: 6732d435f014ee495567a304
paper_authors.name: Weiming Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-29T21:10:08.592Z
paper_authors._id: 64405a9d518271b0d1beea38
paper_authors.avatarUrl: /avatars/b702474588fd7090773320422417a582.svg
paper_authors.fullname: Weiming Ren
paper_authors.type: user
paper_authors.user: wren93
paper_authors._id: 6732d435f014ee495567a305
paper_authors.name: Xinrun Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:26:50.357Z
paper_authors._id: 654907a4a1faff97850c4eff
paper_authors.avatarUrl: /avatars/458c90151614bc7f116943b6e67d6b8a.svg
paper_authors.fullname: du
paper_authors.type: user
paper_authors.user: dododododo
paper_authors._id: 6732d435f014ee495567a306
paper_authors.name: Ge Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-21T21:33:10.345Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 6732d435f014ee495567a307
paper_authors.name: Wenhu Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-12T04:06:18.315Z
paper_authors._id: 6313a86154e6e5d9f0f94e04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
paper_authors.fullname: Wenhu Chen
paper_authors.type: user
paper_authors.user: wenhu
paper_title: OmniEdit: Building Image Editing Generalist Models Through Specialist
    Supervision
paper_summary: Instruction-guided image editing methods have demonstrated significant
  potential by training diffusion models on automatically synthesized or manually
  annotated image editing pairs. However, these methods remain far from
  practical, real-life applications. We identify three primary challenges
  contributing to this gap. Firstly, existing models have limited editing skills
  due to the biased synthesis process. Secondly, these methods are trained with
  datasets with a high volume of noise and artifacts. This is due to the
  application of simple filtering methods like CLIP-score. Thirdly, all these
  datasets are restricted to a single low resolution and fixed aspect ratio,
  limiting the versatility to handle real-world use cases. In this paper, we
  present \omniedit, which is an omnipotent editor to handle seven different
  image editing tasks with any aspect ratio seamlessly. Our contribution is in
  four folds: (1) \omniedit is trained by utilizing the supervision from seven
  different specialist models to ensure task coverage. (2) we utilize importance
  sampling based on the scores provided by large multimodal models (like GPT-4o)
  instead of CLIP-score to improve the data quality. (3) we propose a new editing
  architecture called EditNet to greatly boost the editing success rate, (4) we
  provide images with different aspect ratios to ensure that our model can handle
  any image in the wild. We have curated a test set containing images of
  different aspect ratios, accompanied by diverse instructions to cover different
  tasks. Both automatic evaluation and human evaluations demonstrate that
  \omniedit can significantly outperform all the existing models. Our code,
  dataset and model will be available at
  https://tiger-ai-lab.github.io/OmniEdit/
paper_ai_summary: An omnipotent image editor trained on multisource supervision, importance sampling, and a versatile architecture excels in multiple tasks with diverse aspect ratios and high data quality.
paper_ai_keywords: diffusion models CLIP-score GPT-4o EditNet aspect ratio image editing tasks automatic evaluation human evaluations