CHARTOM: A Visual Theory-of-Mind Benchmark for Multimodal Large Language Models
<|reference_start|>CHARTOM: A Visual Theory-of-Mind Benchmark for Multimodal Large Language Models: We introduce CHARTOM, a visual theory-of-mind benchmark for multimodal large language models. CHARTOM consists of specially designed data visualizing charts. Given a chart, a language model needs to not only correctly comprehend the chart (the FACT question) but also judge if the chart will be misleading to a human reader (the MIND question). Both questions have significant societal benefits. We detail the construction of the CHARTOM benchmark including its calibration on human performance.<|reference_end|>
@article{bharti2024chartom:,
    title={CHARTOM: A Visual Theory-of-Mind Benchmark for Multimodal Large Language
  Models},
    author={Shubham Bharti, Shiyun Cheng, Jihyun Rho, Martina Rao, Xiaojin Zhu},
    journal={arXiv preprint arXiv:2408.14419},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.14419},
    primaryClass={cs.AI cs.CL cs.CV}
}