LEGION: Learning to Ground and Explain for Synthetic Image Detection
The rapid advancements in generative technology have emerged as a
double-edged sword. While offering powerful tools that enhance convenience,
they also pose significant social concerns. As defenders, current synthetic
image detection methods often lack artifact-level textual interpretability and
are overly focused on image manipulation detection, and current datasets
usually suffer from outdated generators and a lack of fine-grained annotations.
In this paper, we introduce SynthScars, a high-quality and diverse dataset
consisting of 12,236 fully synthetic images with human-expert annotations. It
features 4 distinct image content types, 3 categories of artifacts, and
fine-grained annotations covering pixel-level segmentation, detailed textual
explanations, and artifact category labels. Furthermore, we propose LEGION
(LEarning to Ground and explain for Synthetic Image detectiON), a multimodal
large language model (MLLM)-based image forgery analysis framework that
integrates artifact detection, segmentation, and explanation. Building upon
this capability, we further explore LEGION as a controller, integrating it into
image refinement pipelines to guide the generation of higher-quality and more
realistic images. Extensive experiments show that LEGION outperforms existing
methods across multiple benchmarks, particularly surpassing the second-best
traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score.
Moreover, the refined images generated under its guidance exhibit stronger
alignment with human preferences. The code, model, and dataset will be
released.
67dbbe8fafd5251fc6b55730
Hengrui Kang
claimed_verified
2025-03-20T10:45:05.833Z
66c598b536a75deef8bb21c0
/avatars/b3721202d01d3c947c36f0beae43ea1c.svg
Hengrui Kang
user
khr0516
67dbbe8fafd5251fc6b55731
Siwei Wen
67dbbe8fafd5251fc6b55732
Zichen Wen
claimed_verified
2025-03-20T10:45:04.004Z
653b8c3e97a4d71d950e2f20
/avatars/b68880022e14556d0be58c69615db3be.svg
Zichen Wen
user
zichenwen
67dbbe8fafd5251fc6b55733
Junyan Ye
admin_assigned
2025-03-20T13:27:18.626Z
66978ee0b8656f6506b4acb2
/avatars/298acb8222e189fce4368985ee5374a1.svg
Junyan Ye
user
Yejy53
67dbbe8fafd5251fc6b55734
Weijia Li
admin_assigned
2025-03-20T13:27:12.349Z
66d5b56c77a026c3d2086a79
/avatars/45da07fd82fd455955faa05b27a6393f.svg
Weijia Li
user
liweijia
67dbbe8fafd5251fc6b55735
Peilin Feng
admin_assigned
2025-03-20T13:27:04.917Z
65f95363ca387c9d45a2d2ad
/avatars/8e2e8ff5f3b8fbd6d24f0f8947ca5ef4.svg
Peilin Feng
user
Sssunset
67dbbe8fafd5251fc6b55736
Baichuan Zhou
admin_assigned
2025-03-20T13:26:58.914Z
646325085897b675c65aea0f
/avatars/28ce7388f9318b49bdd0a5594c0f6732.svg
Baichuan Zhou
user
bczhou
67dbbe8fafd5251fc6b55737
Bin Wang
67dbbe8fafd5251fc6b55738
Dahua Lin
admin_assigned
2025-03-20T13:26:52.306Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
67dbbe8fafd5251fc6b55739
Linfeng Zhang
admin_assigned
2025-03-20T13:26:43.997Z
642ec9831d1737803dc1c30a
/avatars/c9ded838bad09004c15a27200e66a108.svg
linfeng zhang
user
linfengZ
67dbbe8fafd5251fc6b5573a
Conghui He
admin_assigned
2025-03-20T13:26:34.087Z
63f9fca8d4349b157a109eec
/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
Conghui He
user
conghui
LEGION: Learning to Ground and Explain for Synthetic Image Detection
The rapid advancements in generative technology have emerged as a
double-edged sword. While offering powerful tools that enhance convenience,
they also pose significant social concerns. As defenders, current synthetic
image detection methods often lack artifact-level textual interpretability and
are overly focused on image manipulation detection, and current datasets
usually suffer from outdated generators and a lack of fine-grained annotations.
In this paper, we introduce SynthScars, a high-quality and diverse dataset
consisting of 12,236 fully synthetic images with human-expert annotations. It
features 4 distinct image content types, 3 categories of artifacts, and
fine-grained annotations covering pixel-level segmentation, detailed textual
explanations, and artifact category labels. Furthermore, we propose LEGION
(LEarning to Ground and explain for Synthetic Image detectiON), a multimodal
large language model (MLLM)-based image forgery analysis framework that
integrates artifact detection, segmentation, and explanation. Building upon
this capability, we further explore LEGION as a controller, integrating it into
image refinement pipelines to guide the generation of higher-quality and more
realistic images. Extensive experiments show that LEGION outperforms existing
methods across multiple benchmarks, particularly surpassing the second-best
traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score.
Moreover, the refined images generated under its guidance exhibit stronger
alignment with human preferences. The code, model, and dataset will be
released.
LEGION, a multimodal large language model-based framework, enhances synthetic image detection and generation through artifact detection, segmentation, and explanation, achieving higher accuracy and realism compared to existing methods.
multimodal large language model image forgery analysis framework artifact detection segmentation explanation SynthScars human-expert annotations image refinement mIoU F1 score