Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models
<|reference_start|>Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models: While humans naturally develop theory of mind (ToM), the capability to understand other people's mental states and beliefs, state-of-the-art large language models (LLMs) underperform on simple ToM benchmarks. We posit that we can extend our understanding of LLMs' ToM abilities by evaluating key human ToM precursors -- perception inference and perception-to-belief inference -- in LLMs. We introduce two datasets, Percept-ToMi and Percept-FANToM, to evaluate these precursory inferences for ToM in LLMs by annotating characters' perceptions on ToMi and FANToM, respectively. Our evaluation of eight state-of-the-art LLMs reveals that the models generally perform well in perception inference while exhibiting limited capability in perception-to-belief inference (e.g., lack of inhibitory control). Based on these results, we present PercepToM, a novel ToM method leveraging LLMs' strong perception inference capability while supplementing their limited perception-to-belief inference. Experimental results demonstrate that PercepToM significantly enhances LLM's performance, especially in false belief scenarios.<|reference_end|>
@article{jung2024perceptions,
    title={Perceptions to Beliefs: Exploring Precursory Inferences for Theory of
  Mind in Large Language Models},
    author={Chani Jung, Dongkwan Kim, Jiho Jin, Jiseon Kim, Yeon Seonwoo, Yejin
  Choi, Alice Oh, Hyunwoo Kim},
    journal={arXiv preprint arXiv:2407.06004},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.06004},
    primaryClass={cs.CL}
}