Decoding News Narratives: A Critical Analysis of Large Language Models in Framing Detection
<|reference_start|>Decoding News Narratives: A Critical Analysis of Large Language Models in Framing Detection: Previous studies on framing have relied on manual analysis or fine-tuning models with limited annotated datasets. However, pre-trained models, with their diverse training backgrounds, offer a promising alternative. This paper presents a comprehensive analysis of GPT-4, GPT-3.5 Turbo, and FLAN-T5 models in detecting framing in news headlines. We evaluated these models in various scenarios: zero-shot, few-shot with in-domain examples, cross-domain examples, and settings where models explain their predictions. Our results show that explainable predictions lead to more reliable outcomes. GPT-4 performed exceptionally well in few-shot settings but often misinterpreted emotional language as framing, highlighting a significant challenge. Additionally, the results suggest that consistent predictions across multiple models could help identify potential annotation inaccuracies in datasets. Finally, we propose a new small dataset for real-world evaluation on headlines from a diverse set of topics.<|reference_end|>
@article{pastorino2024decoding,
    title={Decoding News Narratives: A Critical Analysis of Large Language Models
  in Framing Detection},
    author={Valeria Pastorino, Jasivan A. Sivakumar, Nafise Sadat Moosavi},
    journal={arXiv preprint arXiv:2402.11621},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.11621},
    primaryClass={cs.CL}
}