title: Humans expect rationality and cooperation from LLM opponents in
    strategic games
summary: As Large Language Models (LLMs) integrate into our social and economic
  interactions, we need to deepen our understanding of how humans respond to LLMs
  opponents in strategic settings. We present the results of the first controlled
  monetarily-incentivised laboratory experiment looking at differences in human
  behaviour in a multi-player p-beauty contest against other humans and LLMs. We
  use a within-subject design in order to compare behaviour at the individual
  level. We show that, in this environment, human subjects choose significantly
  lower numbers when playing against LLMs than humans, which is mainly driven by
  the increased prevalence of `zero' Nash-equilibrium choices. This shift is
  mainly driven by subjects with high strategic reasoning ability. Subjects who
  play the zero Nash-equilibrium choice motivate their strategy by appealing to
  perceived LLM's reasoning ability and, unexpectedly, propensity towards
  cooperation. Our findings provide foundational insights into the multi-player
  human-LLM interaction in simultaneous choice games, uncover heterogeneities in
  both subjects' behaviour and beliefs about LLM's play when playing against
  them, and suggest important implications for mechanism design in mixed
  human-LLM systems.
paper_authors._id: 682ae098730bd40a0755f87c
paper_authors.name: Darija Barak
paper_authors._id: 682ae098730bd40a0755f87d
paper_authors.name: Miguel Costa-Gomes
paper_title: Humans expect rationality and cooperation from LLM opponents in
    strategic games
paper_summary: As Large Language Models (LLMs) integrate into our social and economic
  interactions, we need to deepen our understanding of how humans respond to LLMs
  opponents in strategic settings. We present the results of the first controlled
  monetarily-incentivised laboratory experiment looking at differences in human
  behaviour in a multi-player p-beauty contest against other humans and LLMs. We
  use a within-subject design in order to compare behaviour at the individual
  level. We show that, in this environment, human subjects choose significantly
  lower numbers when playing against LLMs than humans, which is mainly driven by
  the increased prevalence of `zero' Nash-equilibrium choices. This shift is
  mainly driven by subjects with high strategic reasoning ability. Subjects who
  play the zero Nash-equilibrium choice motivate their strategy by appealing to
  perceived LLM's reasoning ability and, unexpectedly, propensity towards
  cooperation. Our findings provide foundational insights into the multi-player
  human-LLM interaction in simultaneous choice games, uncover heterogeneities in
  both subjects' behaviour and beliefs about LLM's play when playing against
  them, and suggest important implications for mechanism design in mixed
  human-LLM systems.
paper_ai_summary: Human subjects in a p-beauty contest choose lower numbers when playing against LLMs, driven by zero Nash-equilibrium choices, which highlights differences in interaction and strategic reasoning.
paper_ai_keywords: p-beauty contest Nash-equilibrium multi-player human-LLM interaction mechanism design