Complex-Edit: CoT-Like Instruction Generation for
  Complexity-Controllable Image Editing Benchmark
We introduce Complex-Edit, a comprehensive benchmark designed to
systematically evaluate instruction-based image editing models across
instructions of varying complexity. To develop this benchmark, we harness
GPT-4o to automatically collect a diverse set of editing instructions at scale.
Our approach follows a well-structured ``Chain-of-Edit'' pipeline: we first
generate individual atomic editing tasks independently and then integrate them
to form cohesive, complex instructions. Additionally, we introduce a suite of
metrics to assess various aspects of editing performance, along with a
VLM-based auto-evaluation pipeline that supports large-scale assessments. Our
benchmark yields several notable insights: 1) Open-source models significantly
underperform relative to proprietary, closed-source models, with the
performance gap widening as instruction complexity increases; 2) Increased
instructional complexity primarily impairs the models' ability to retain key
elements from the input images and to preserve the overall aesthetic quality;
3) Decomposing a complex instruction into a sequence of atomic steps, executed
in a step-by-step manner, substantially degrades performance across multiple
metrics; 4) A straightforward Best-of-N selection strategy improves results for
both direct editing and the step-by-step sequential approach; and 5) We observe
a ``curse of synthetic data'': when synthetic data is involved in model
training, the edited images from such models tend to appear increasingly
synthetic as the complexity of the editing instructions rises -- a phenomenon
that intriguingly also manifests in the latest GPT-4o outputs.
6802d28a5c4268810bdd148e
Siwei Yang
claimed_verified
2025-04-22T10:13:17.157Z
6433fd1095b8ab049385d2b3
/avatars/718117cd4631b7c297add7e559032b2f.svg
Siwei Yang
user
LAOS-Y
6802d28a5c4268810bdd148f
Mude Hui
6802d28a5c4268810bdd1490
Bingchen Zhao
claimed_verified
2025-04-20T15:02:15.504Z
62dcd71075e9787ec5aa41ba
/avatars/f37ce036b76180ed0fa004f9c8c09363.svg
Bingchen Zhao
user
tennant
6802d28a5c4268810bdd1491
Yuyin Zhou
6802d28a5c4268810bdd1492
Nataniel Ruiz
6802d28a5c4268810bdd1493
Cihang Xie
claimed_verified
2025-07-31T08:16:39.025Z
645eb61da3c5cd8a16efffff
/avatars/9112bfeed598dfabf9e077e69e09ecc9.svg
Cihang Xie
user
cihangxie
Complex-Edit: CoT-Like Instruction Generation for
  Complexity-Controllable Image Editing Benchmark
We introduce Complex-Edit, a comprehensive benchmark designed to
systematically evaluate instruction-based image editing models across
instructions of varying complexity. To develop this benchmark, we harness
GPT-4o to automatically collect a diverse set of editing instructions at scale.
Our approach follows a well-structured ``Chain-of-Edit'' pipeline: we first
generate individual atomic editing tasks independently and then integrate them
to form cohesive, complex instructions. Additionally, we introduce a suite of
metrics to assess various aspects of editing performance, along with a
VLM-based auto-evaluation pipeline that supports large-scale assessments. Our
benchmark yields several notable insights: 1) Open-source models significantly
underperform relative to proprietary, closed-source models, with the
performance gap widening as instruction complexity increases; 2) Increased
instructional complexity primarily impairs the models' ability to retain key
elements from the input images and to preserve the overall aesthetic quality;
3) Decomposing a complex instruction into a sequence of atomic steps, executed
in a step-by-step manner, substantially degrades performance across multiple
metrics; 4) A straightforward Best-of-N selection strategy improves results for
both direct editing and the step-by-step sequential approach; and 5) We observe
a ``curse of synthetic data'': when synthetic data is involved in model
training, the edited images from such models tend to appear increasingly
synthetic as the complexity of the editing instructions rises -- a phenomenon
that intriguingly also manifests in the latest GPT-4o outputs.
A benchmark using GPT-4o evaluates instruction-based image editing models across varying complexities, revealing performance gaps, retention issues, and synthetic data challenges.
GPT-4o Chain-of-Edit VLM-based auto-evaluation Best-of-N selection strategy