Can GPT-4 Models Detect Misleading Visualizations?
<|reference_start|>Can GPT-4 Models Detect Misleading Visualizations?: The proliferation of misleading visualizations online, particularly during critical events like public health crises and elections, poses a significant risk. This study investigates the capability of GPT-4 models (4V, 4o, and 4o mini) to detect misleading visualizations. Utilizing a dataset of tweet-visualization pairs containing various visual misleaders, we test these models under four experimental conditions with different levels of guidance. We show that GPT-4 models can detect misleading visualizations with moderate accuracy without prior training (naive zero-shot) and that performance notably improves when provided with definitions of misleaders (guided zero-shot). However, a single prompt engineering technique does not yield the best results for all misleader types. Specifically, providing the models with misleader definitions and examples (guided few-shot) proves more effective for reasoning misleaders, while guided zero-shot performs better for design misleaders. This study underscores the feasibility of using large vision-language models to detect visual misinformation and the importance of prompt engineering for optimized detection accuracy.<|reference_end|>
@article{alexander2024can,
    title={Can GPT-4 Models Detect Misleading Visualizations?},
    author={Jason Alexander, Priyal Nanda, Kai-Cheng Yang, Ali Sarvghad},
    journal={arXiv preprint arXiv:2408.12617},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.12617},
    primaryClass={cs.CV cs.CY cs.SI}
}