title: Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal
    Evolution of Human States
summary: As Large Language Models (LLMs) increasingly participate in human-AI
  interactions, evaluating their Theory of Mind (ToM) capabilities - particularly
  their ability to track dynamic mental states - becomes crucial. While existing
  benchmarks assess basic ToM abilities, they predominantly focus on static
  snapshots of mental states, overlooking the temporal evolution that
  characterizes real-world social interactions. We present DynToM, a
  novel benchmark specifically designed to evaluate LLMs' ability to understand
  and track the temporal progression of mental states across interconnected
  scenarios. Through a systematic four-step framework, we generate 1,100 social
  contexts encompassing 5,500 scenarios and 78,100 questions, each validated for
  realism and quality. Our comprehensive evaluation of ten state-of-the-art LLMs
  reveals that their average performance underperforms humans by 44.7\%, with
  performance degrading significantly when tracking and reasoning about the shift
  of mental states. This performance gap highlights fundamental limitations in
  current LLMs' ability to model the dynamic nature of human mental states.
paper_authors._id: 6833c6ff97966d18e7b995b0
paper_authors.name: Yang Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T19:14:41.639Z
paper_authors._id: 6002c316698168af3bb9f4a6
paper_authors.avatarUrl: /avatars/1028860b59951be04e94b126b0985dc0.svg
paper_authors.fullname: yangxiao
paper_authors.type: user
paper_authors.user: YangXiao-nlp
paper_authors._id: 6833c6ff97966d18e7b995b1
paper_authors.name: Jiashuo Wang
paper_authors._id: 6833c6ff97966d18e7b995b2
paper_authors.name: Qiancheng Xu
paper_authors._id: 6833c6ff97966d18e7b995b3
paper_authors.name: Changhe Song
paper_authors._id: 6833c6ff97966d18e7b995b4
paper_authors.name: Chunpu Xu
paper_authors._id: 6833c6ff97966d18e7b995b5
paper_authors.name: Yi Cheng
paper_authors._id: 6833c6ff97966d18e7b995b6
paper_authors.name: Wenjie Li
paper_authors._id: 6833c6ff97966d18e7b995b7
paper_authors.name: Pengfei Liu
paper_title: Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal
    Evolution of Human States
paper_summary: As Large Language Models (LLMs) increasingly participate in human-AI
  interactions, evaluating their Theory of Mind (ToM) capabilities - particularly
  their ability to track dynamic mental states - becomes crucial. While existing
  benchmarks assess basic ToM abilities, they predominantly focus on static
  snapshots of mental states, overlooking the temporal evolution that
  characterizes real-world social interactions. We present DynToM, a
  novel benchmark specifically designed to evaluate LLMs' ability to understand
  and track the temporal progression of mental states across interconnected
  scenarios. Through a systematic four-step framework, we generate 1,100 social
  contexts encompassing 5,500 scenarios and 78,100 questions, each validated for
  realism and quality. Our comprehensive evaluation of ten state-of-the-art LLMs
  reveals that their average performance underperforms humans by 44.7\%, with
  performance degrading significantly when tracking and reasoning about the shift
  of mental states. This performance gap highlights fundamental limitations in
  current LLMs' ability to model the dynamic nature of human mental states.
paper_ai_summary: The DynToM benchmark evaluates LLMs' ability to track and understand the temporal progression of mental states, revealing significant gaps compared to human performance.
paper_ai_keywords: Large Language Models (LLMs) Theory of Mind (ToM) DynToM social contexts mental states temporal progression evaluation framework state-of-the-art LLMs performance gap dynamic mental states