RefEdit: A Benchmark and Method for Improving Instruction-based Image
  Editing Model on Referring Expressions
Despite recent advances in inversion and instruction-based image editing,
existing approaches primarily excel at editing single, prominent objects but
significantly struggle when applied to complex scenes containing multiple
entities. To quantify this gap, we first introduce RefEdit-Bench, a rigorous
real-world benchmark rooted in RefCOCO, where even baselines trained on
millions of samples perform poorly. To overcome this limitation, we introduce
RefEdit -- an instruction-based editing model trained on our scalable synthetic
data generation pipeline. Our RefEdit, trained on only 20,000 editing triplets,
outperforms the Flux/SD3 model-based baselines trained on millions of data.
Extensive evaluations across various benchmarks demonstrate that our model not
only excels in referring expression tasks but also enhances performance on
traditional benchmarks, achieving state-of-the-art results comparable to
closed-source methods. We release data \& checkpoint for reproducibility.
684105479060432bf302b432
Bimsara Pathiraja
684105479060432bf302b433
Maitreya Patel
claimed_verified
2025-06-05T07:26:48.135Z
622d2ff38d04fd29a9ccf1a7
https://cdn-avatars.huggingface.co/v1/production/uploads/622d2ff38d04fd29a9ccf1a7/ORpGrlU8Lm_oSEVftcZEK.jpeg
Maitreya Patel
user
mpatel57
684105479060432bf302b434
Shivam Singh
684105479060432bf302b435
Yezhou Yang
684105479060432bf302b436
Chitta Baral
RefEdit: A Benchmark and Method for Improving Instruction-based Image
  Editing Model on Referring Expressions
Despite recent advances in inversion and instruction-based image editing,
existing approaches primarily excel at editing single, prominent objects but
significantly struggle when applied to complex scenes containing multiple
entities. To quantify this gap, we first introduce RefEdit-Bench, a rigorous
real-world benchmark rooted in RefCOCO, where even baselines trained on
millions of samples perform poorly. To overcome this limitation, we introduce
RefEdit -- an instruction-based editing model trained on our scalable synthetic
data generation pipeline. Our RefEdit, trained on only 20,000 editing triplets,
outperforms the Flux/SD3 model-based baselines trained on millions of data.
Extensive evaluations across various benchmarks demonstrate that our model not
only excels in referring expression tasks but also enhances performance on
traditional benchmarks, achieving state-of-the-art results comparable to
closed-source methods. We release data \& checkpoint for reproducibility.
RefEdit, an instruction-based editing model trained on synthetic data, outperforms baselines in complex scene editing and referring expression tasks.
RefEdit-Bench RefCOCO instruction-based editing model scalable synthetic data generation pipeline Flux/SD3 state-of-the-art results