title: How FaR Are Large Language Models From Agents with Theory-of-Mind?
summary: "Thinking is for Doing." Humans can infer other people's mental states from
  observations--an ability called Theory-of-Mind (ToM)--and subsequently act
  pragmatically on those inferences. Existing question answering benchmarks such
  as ToMi ask models questions to make inferences about beliefs of characters in
  a story, but do not test whether models can then use these inferences to guide
  their actions. We propose a new evaluation paradigm for large language models
  (LLMs): Thinking for Doing (T4D), which requires models to connect inferences
  about others' mental states to actions in social scenarios. Experiments on T4D
  demonstrate that LLMs such as GPT-4 and PaLM 2 seemingly excel at tracking
  characters' beliefs in stories, but they struggle to translate this capability
  into strategic action. Our analysis reveals the core challenge for LLMs lies in
  identifying the implicit inferences about mental states without being
  explicitly asked about as in ToMi, that lead to choosing the correct action in
  T4D. To bridge this gap, we introduce a zero-shot prompting framework, Foresee
  and Reflect (FaR), which provides a reasoning structure that encourages LLMs to
  anticipate future challenges and reason about potential actions. FaR boosts
  GPT-4's performance from 50% to 71% on T4D, outperforming other prompting
  methods such as Chain-of-Thought and Self-Ask. Moreover, FaR generalizes to
  diverse out-of-distribution story structures and scenarios that also require
  ToM inferences to choose an action, consistently outperforming other methods
  including few-shot in-context learning.
paper_authors._id: 651f60b4f23fc101b16196c1
paper_authors.name: Pei Zhou
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-10-06T01:19:51.694Z
paper_authors._id: 60844fd056f7623a0b84914b
paper_authors.avatarUrl: /avatars/f7d2763258b8c614783ead9480fbcf8d.svg
paper_authors.fullname: Pei Zhou
paper_authors.type: user
paper_authors.user: peiz
paper_authors._id: 651f60b4f23fc101b16196c2
paper_authors.name: Aman Madaan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:32:58.731Z
paper_authors._id: 61bbc2b2d96f5d4111901dcb
paper_authors.avatarUrl: /avatars/83c826d5cad7bab27d33585cb7e64575.svg
paper_authors.fullname: Aman Madaan
paper_authors.type: user
paper_authors.user: amanmadaan
paper_authors._id: 651f60b4f23fc101b16196c3
paper_authors.name: Srividya Pranavi Potharaju
paper_authors._id: 651f60b4f23fc101b16196c4
paper_authors.name: Aditya Gupta
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-21T09:22:55.725Z
paper_authors._id: 64bac8eb01f1983a862e58b2
paper_authors.avatarUrl: /avatars/ba005db511bf9fa2e79ca2e78852f8fd.svg
paper_authors.fullname: Aditya Gupta
paper_authors.type: user
paper_authors.user: aditya2211
paper_authors._id: 651f60b4f23fc101b16196c5
paper_authors.name: Kevin R. McKee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:33:40.074Z
paper_authors._id: 624911a1edf42bf849d0ffc5
paper_authors.avatarUrl: /avatars/fdf70a412b2b90d0407dc6adb484319f.svg
paper_authors.fullname: Kevin Mckee
paper_authors.type: user
paper_authors.user: Kmckee95
paper_authors._id: 651f60b4f23fc101b16196c6
paper_authors.name: Ari Holtzman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-11T07:34:49.773Z
paper_authors._id: 62a930029e44ab4160814c07
paper_authors.avatarUrl: /avatars/16eafa37a14715dd286d2670af88a0d0.svg
paper_authors.fullname: Ari Holtzman
paper_authors.type: user
paper_authors.user: ari-holtzman
paper_authors._id: 651f60b4f23fc101b16196c7
paper_authors.name: Jay Pujara
paper_authors._id: 651f60b4f23fc101b16196c8
paper_authors.name: Xiang Ren
paper_authors._id: 651f60b4f23fc101b16196c9
paper_authors.name: Swaroop Mishra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-09T08:35:19.039Z
paper_authors._id: 61841feb7c380c66d40cad54
paper_authors.avatarUrl: /avatars/7a79362a24b844860781fb0625527eae.svg
paper_authors.fullname: Swaroop Mishra
paper_authors.type: user
paper_authors.user: swaroop7
paper_authors._id: 651f60b4f23fc101b16196ca
paper_authors.name: Aida Nematzadeh
paper_authors._id: 651f60b4f23fc101b16196cb
paper_authors.name: Shyam Upadhyay
paper_authors._id: 651f60b4f23fc101b16196cc
paper_authors.name: Manaal Faruqui
paper_title: How FaR Are Large Language Models From Agents with Theory-of-Mind?
paper_summary: "Thinking is for Doing." Humans can infer other people's mental states from
  observations--an ability called Theory-of-Mind (ToM)--and subsequently act
  pragmatically on those inferences. Existing question answering benchmarks such
  as ToMi ask models questions to make inferences about beliefs of characters in
  a story, but do not test whether models can then use these inferences to guide
  their actions. We propose a new evaluation paradigm for large language models
  (LLMs): Thinking for Doing (T4D), which requires models to connect inferences
  about others' mental states to actions in social scenarios. Experiments on T4D
  demonstrate that LLMs such as GPT-4 and PaLM 2 seemingly excel at tracking
  characters' beliefs in stories, but they struggle to translate this capability
  into strategic action. Our analysis reveals the core challenge for LLMs lies in
  identifying the implicit inferences about mental states without being
  explicitly asked about as in ToMi, that lead to choosing the correct action in
  T4D. To bridge this gap, we introduce a zero-shot prompting framework, Foresee
  and Reflect (FaR), which provides a reasoning structure that encourages LLMs to
  anticipate future challenges and reason about potential actions. FaR boosts
  GPT-4's performance from 50% to 71% on T4D, outperforming other prompting
  methods such as Chain-of-Thought and Self-Ask. Moreover, FaR generalizes to
  diverse out-of-distribution story structures and scenarios that also require
  ToM inferences to choose an action, consistently outperforming other methods
  including few-shot in-context learning.
paper_ai_summary: LLMs perform well in inferring beliefs but struggle translating these into actions; Foresee and Reflect (FaR) improves their action guidance in social scenarios.
paper_ai_keywords: Theory-of-Mind (ToM) large language models (LLMs) Thinking for Doing (T4D) mental states strategic action zero-shot prompting Chain-of-Thought Self-Ask