title: RefEdit: A Benchmark and Method for Improving Instruction-based Image
    Editing Model on Referring Expressions
summary: Despite recent advances in inversion and instruction-based image editing,
  existing approaches primarily excel at editing single, prominent objects but
  significantly struggle when applied to complex scenes containing multiple
  entities. To quantify this gap, we first introduce RefEdit-Bench, a rigorous
  real-world benchmark rooted in RefCOCO, where even baselines trained on
  millions of samples perform poorly. To overcome this limitation, we introduce
  RefEdit -- an instruction-based editing model trained on our scalable synthetic
  data generation pipeline. Our RefEdit, trained on only 20,000 editing triplets,
  outperforms the Flux/SD3 model-based baselines trained on millions of data.
  Extensive evaluations across various benchmarks demonstrate that our model not
  only excels in referring expression tasks but also enhances performance on
  traditional benchmarks, achieving state-of-the-art results comparable to
  closed-source methods. We release data \& checkpoint for reproducibility.
paper_authors._id: 684105479060432bf302b432
paper_authors.name: Bimsara Pathiraja
paper_authors._id: 684105479060432bf302b433
paper_authors.name: Maitreya Patel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:26:48.135Z
paper_authors._id: 622d2ff38d04fd29a9ccf1a7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/622d2ff38d04fd29a9ccf1a7/ORpGrlU8Lm_oSEVftcZEK.jpeg
paper_authors.fullname: Maitreya Patel
paper_authors.type: user
paper_authors.user: mpatel57
paper_authors._id: 684105479060432bf302b434
paper_authors.name: Shivam Singh
paper_authors._id: 684105479060432bf302b435
paper_authors.name: Yezhou Yang
paper_authors._id: 684105479060432bf302b436
paper_authors.name: Chitta Baral
paper_title: RefEdit: A Benchmark and Method for Improving Instruction-based Image
    Editing Model on Referring Expressions
paper_summary: Despite recent advances in inversion and instruction-based image editing,
  existing approaches primarily excel at editing single, prominent objects but
  significantly struggle when applied to complex scenes containing multiple
  entities. To quantify this gap, we first introduce RefEdit-Bench, a rigorous
  real-world benchmark rooted in RefCOCO, where even baselines trained on
  millions of samples perform poorly. To overcome this limitation, we introduce
  RefEdit -- an instruction-based editing model trained on our scalable synthetic
  data generation pipeline. Our RefEdit, trained on only 20,000 editing triplets,
  outperforms the Flux/SD3 model-based baselines trained on millions of data.
  Extensive evaluations across various benchmarks demonstrate that our model not
  only excels in referring expression tasks but also enhances performance on
  traditional benchmarks, achieving state-of-the-art results comparable to
  closed-source methods. We release data \& checkpoint for reproducibility.
paper_ai_summary: RefEdit, an instruction-based editing model trained on synthetic data, outperforms baselines in complex scene editing and referring expression tasks.
paper_ai_keywords: RefEdit-Bench RefCOCO instruction-based editing model scalable synthetic data generation pipeline Flux/SD3 state-of-the-art results