MiRAGeNews: Multimodal Realistic AI-Generated News Detection
The proliferation of inflammatory or misleading "fake" news content has
become increasingly common in recent years. Simultaneously, it has become
easier than ever to use AI tools to generate photorealistic images depicting
any scene imaginable. Combining these two -- AI-generated fake news content --
is particularly potent and dangerous. To combat the spread of AI-generated fake
news, we propose the MiRAGeNews Dataset, a dataset of 12,500 high-quality real
and AI-generated image-caption pairs from state-of-the-art generators. We find
that our dataset poses a significant challenge to humans (60% F-1) and
state-of-the-art multi-modal LLMs (< 24% F-1). Using our dataset we train a
multi-modal detector (MiRAGe) that improves by +5.1% F-1 over state-of-the-art
baselines on image-caption pairs from out-of-domain image generators and news
publishers. We release our code and data to aid future work on detecting
AI-generated content.
670d33df6f346132013d7214
Runsheng Huang
670d33df6f346132013d7215
Liam Dugan
claimed_verified
2024-10-14T15:30:29.739Z
6509e1e01b3694179dee256e
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sUbok2yloLElaebkps3lT.png
Liam Dugan
user
liamdugan
670d33df6f346132013d7216
Yue Yang
claimed_verified
2025-02-21T15:06:17.308Z
62f6c68904e5e02f82b04690
https://cdn-avatars.huggingface.co/v1/production/uploads/62f6c68904e5e02f82b04690/kK2-PkeAGzAOLhkfajswf.jpeg
Yue Yang
user
yyupenn
670d33df6f346132013d7217
Chris Callison-Burch
MiRAGeNews: Multimodal Realistic AI-Generated News Detection
The proliferation of inflammatory or misleading "fake" news content has
become increasingly common in recent years. Simultaneously, it has become
easier than ever to use AI tools to generate photorealistic images depicting
any scene imaginable. Combining these two -- AI-generated fake news content --
is particularly potent and dangerous. To combat the spread of AI-generated fake
news, we propose the MiRAGeNews Dataset, a dataset of 12,500 high-quality real
and AI-generated image-caption pairs from state-of-the-art generators. We find
that our dataset poses a significant challenge to humans (60% F-1) and
state-of-the-art multi-modal LLMs (< 24% F-1). Using our dataset we train a
multi-modal detector (MiRAGe) that improves by +5.1% F-1 over state-of-the-art
baselines on image-caption pairs from out-of-domain image generators and news
publishers. We release our code and data to aid future work on detecting
AI-generated content.
A multi-modal detector (MiRAGe) is trained using the MiRAGeNews dataset to improve detection of AI-generated image-caption pairs, outperforming existing methods.
multi-modal detector multi-modal LLMs image-caption pairs state-of-the-art generators out-of-domain image generators fake news