title: LEGION: Learning to Ground and Explain for Synthetic Image Detection
summary: The rapid advancements in generative technology have emerged as a
  double-edged sword. While offering powerful tools that enhance convenience,
  they also pose significant social concerns. As defenders, current synthetic
  image detection methods often lack artifact-level textual interpretability and
  are overly focused on image manipulation detection, and current datasets
  usually suffer from outdated generators and a lack of fine-grained annotations.
  In this paper, we introduce SynthScars, a high-quality and diverse dataset
  consisting of 12,236 fully synthetic images with human-expert annotations. It
  features 4 distinct image content types, 3 categories of artifacts, and
  fine-grained annotations covering pixel-level segmentation, detailed textual
  explanations, and artifact category labels. Furthermore, we propose LEGION
  (LEarning to Ground and explain for Synthetic Image detectiON), a multimodal
  large language model (MLLM)-based image forgery analysis framework that
  integrates artifact detection, segmentation, and explanation. Building upon
  this capability, we further explore LEGION as a controller, integrating it into
  image refinement pipelines to guide the generation of higher-quality and more
  realistic images. Extensive experiments show that LEGION outperforms existing
  methods across multiple benchmarks, particularly surpassing the second-best
  traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score.
  Moreover, the refined images generated under its guidance exhibit stronger
  alignment with human preferences. The code, model, and dataset will be
  released.
paper_authors._id: 67dbbe8fafd5251fc6b55730
paper_authors.name: Hengrui Kang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:45:05.833Z
paper_authors._id: 66c598b536a75deef8bb21c0
paper_authors.avatarUrl: /avatars/b3721202d01d3c947c36f0beae43ea1c.svg
paper_authors.fullname: Hengrui Kang
paper_authors.type: user
paper_authors.user: khr0516
paper_authors._id: 67dbbe8fafd5251fc6b55731
paper_authors.name: Siwei Wen
paper_authors._id: 67dbbe8fafd5251fc6b55732
paper_authors.name: Zichen Wen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:45:04.004Z
paper_authors._id: 653b8c3e97a4d71d950e2f20
paper_authors.avatarUrl: /avatars/b68880022e14556d0be58c69615db3be.svg
paper_authors.fullname: Zichen Wen
paper_authors.type: user
paper_authors.user: zichenwen
paper_authors._id: 67dbbe8fafd5251fc6b55733
paper_authors.name: Junyan Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:27:18.626Z
paper_authors._id: 66978ee0b8656f6506b4acb2
paper_authors.avatarUrl: /avatars/298acb8222e189fce4368985ee5374a1.svg
paper_authors.fullname: Junyan Ye
paper_authors.type: user
paper_authors.user: Yejy53
paper_authors._id: 67dbbe8fafd5251fc6b55734
paper_authors.name: Weijia Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:27:12.349Z
paper_authors._id: 66d5b56c77a026c3d2086a79
paper_authors.avatarUrl: /avatars/45da07fd82fd455955faa05b27a6393f.svg
paper_authors.fullname: Weijia Li
paper_authors.type: user
paper_authors.user: liweijia
paper_authors._id: 67dbbe8fafd5251fc6b55735
paper_authors.name: Peilin Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:27:04.917Z
paper_authors._id: 65f95363ca387c9d45a2d2ad
paper_authors.avatarUrl: /avatars/8e2e8ff5f3b8fbd6d24f0f8947ca5ef4.svg
paper_authors.fullname: Peilin Feng
paper_authors.type: user
paper_authors.user: Sssunset
paper_authors._id: 67dbbe8fafd5251fc6b55736
paper_authors.name: Baichuan Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:26:58.914Z
paper_authors._id: 646325085897b675c65aea0f
paper_authors.avatarUrl: /avatars/28ce7388f9318b49bdd0a5594c0f6732.svg
paper_authors.fullname: Baichuan Zhou
paper_authors.type: user
paper_authors.user: bczhou
paper_authors._id: 67dbbe8fafd5251fc6b55737
paper_authors.name: Bin Wang
paper_authors._id: 67dbbe8fafd5251fc6b55738
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:26:52.306Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 67dbbe8fafd5251fc6b55739
paper_authors.name: Linfeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:26:43.997Z
paper_authors._id: 642ec9831d1737803dc1c30a
paper_authors.avatarUrl: /avatars/c9ded838bad09004c15a27200e66a108.svg
paper_authors.fullname: linfeng zhang
paper_authors.type: user
paper_authors.user: linfengZ
paper_authors._id: 67dbbe8fafd5251fc6b5573a
paper_authors.name: Conghui He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T13:26:34.087Z
paper_authors._id: 63f9fca8d4349b157a109eec
paper_authors.avatarUrl: /avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
paper_authors.fullname: Conghui He
paper_authors.type: user
paper_authors.user: conghui
paper_title: LEGION: Learning to Ground and Explain for Synthetic Image Detection
paper_summary: The rapid advancements in generative technology have emerged as a
  double-edged sword. While offering powerful tools that enhance convenience,
  they also pose significant social concerns. As defenders, current synthetic
  image detection methods often lack artifact-level textual interpretability and
  are overly focused on image manipulation detection, and current datasets
  usually suffer from outdated generators and a lack of fine-grained annotations.
  In this paper, we introduce SynthScars, a high-quality and diverse dataset
  consisting of 12,236 fully synthetic images with human-expert annotations. It
  features 4 distinct image content types, 3 categories of artifacts, and
  fine-grained annotations covering pixel-level segmentation, detailed textual
  explanations, and artifact category labels. Furthermore, we propose LEGION
  (LEarning to Ground and explain for Synthetic Image detectiON), a multimodal
  large language model (MLLM)-based image forgery analysis framework that
  integrates artifact detection, segmentation, and explanation. Building upon
  this capability, we further explore LEGION as a controller, integrating it into
  image refinement pipelines to guide the generation of higher-quality and more
  realistic images. Extensive experiments show that LEGION outperforms existing
  methods across multiple benchmarks, particularly surpassing the second-best
  traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score.
  Moreover, the refined images generated under its guidance exhibit stronger
  alignment with human preferences. The code, model, and dataset will be
  released.
paper_ai_summary: LEGION, a multimodal large language model-based framework, enhances synthetic image detection and generation through artifact detection, segmentation, and explanation, achieving higher accuracy and realism compared to existing methods.
paper_ai_keywords: multimodal large language model image forgery analysis framework artifact detection segmentation explanation SynthScars human-expert annotations image refinement mIoU F1 score