title: Playing repeated games with Large Language Models
summary: Large Language Models (LLMs) are transforming society and permeating into
  diverse applications. As a result, LLMs will frequently interact with us and
  other agents. It is, therefore, of great societal value to understand how LLMs
  behave in interactive social settings. Here, we propose to use behavioral game
  theory to study LLM's cooperation and coordination behavior. To do so, we let
  different LLMs (GPT-3, GPT-3.5, and GPT-4) play finitely repeated games with
  each other and with other, human-like strategies. Our results show that LLMs
  generally perform well in such tasks and also uncover persistent behavioral
  signatures. In a large set of two players-two strategies games, we find that
  LLMs are particularly good at games where valuing their own self-interest pays
  off, like the iterated Prisoner's Dilemma family. However, they behave
  sub-optimally in games that require coordination. We, therefore, further focus
  on two games from these distinct families. In the canonical iterated Prisoner's
  Dilemma, we find that GPT-4 acts particularly unforgivingly, always defecting
  after another agent has defected only once. In the Battle of the Sexes, we find
  that GPT-4 cannot match the behavior of the simple convention to alternate
  between options. We verify that these behavioral signatures are stable across
  robustness checks. Finally, we show how GPT-4's behavior can be modified by
  providing further information about the other player as well as by asking it to
  predict the other player's actions before making a choice. These results enrich
  our understanding of LLM's social behavior and pave the way for a behavioral
  game theory for machines.
paper_authors._id: 64740d8916ef9c21dbaa09ad
paper_authors.name: Elif Akata
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-30T13:21:58.670Z
paper_authors._id: 6369176764aad59d4d3e9e9d
paper_authors.avatarUrl: /avatars/1b501f1fb135de030d603e10c60a8b48.svg
paper_authors.fullname: Elif Akata
paper_authors.type: user
paper_authors.user: eakata
paper_authors._id: 64740d8916ef9c21dbaa09ae
paper_authors.name: Lion Schulz
paper_authors._id: 64740d8916ef9c21dbaa09af
paper_authors.name: Julian Coda-Forno
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-30T13:22:25.770Z
paper_authors._id: 6475c4694a0bae093fe222df
paper_authors.avatarUrl: /avatars/938c6268e732086905df724477625df8.svg
paper_authors.fullname: Julian Coda-Forno
paper_authors.type: user
paper_authors.user: Julian-CF
paper_authors._id: 64740d8916ef9c21dbaa09b0
paper_authors.name: Seong Joon Oh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-30T13:22:59.885Z
paper_authors._id: 638a50450f10aa3064f03f23
paper_authors.avatarUrl: /avatars/0c068458e42950c851758a238225c3a6.svg
paper_authors.fullname: Seong Joon Oh
paper_authors.type: user
paper_authors.user: coallaoh
paper_authors._id: 64740d8916ef9c21dbaa09b1
paper_authors.name: Matthias Bethge
paper_authors._id: 64740d8916ef9c21dbaa09b2
paper_authors.name: Eric Schulz
paper_title: Playing repeated games with Large Language Models
paper_summary: Large Language Models (LLMs) are transforming society and permeating into
  diverse applications. As a result, LLMs will frequently interact with us and
  other agents. It is, therefore, of great societal value to understand how LLMs
  behave in interactive social settings. Here, we propose to use behavioral game
  theory to study LLM's cooperation and coordination behavior. To do so, we let
  different LLMs (GPT-3, GPT-3.5, and GPT-4) play finitely repeated games with
  each other and with other, human-like strategies. Our results show that LLMs
  generally perform well in such tasks and also uncover persistent behavioral
  signatures. In a large set of two players-two strategies games, we find that
  LLMs are particularly good at games where valuing their own self-interest pays
  off, like the iterated Prisoner's Dilemma family. However, they behave
  sub-optimally in games that require coordination. We, therefore, further focus
  on two games from these distinct families. In the canonical iterated Prisoner's
  Dilemma, we find that GPT-4 acts particularly unforgivingly, always defecting
  after another agent has defected only once. In the Battle of the Sexes, we find
  that GPT-4 cannot match the behavior of the simple convention to alternate
  between options. We verify that these behavioral signatures are stable across
  robustness checks. Finally, we show how GPT-4's behavior can be modified by
  providing further information about the other player as well as by asking it to
  predict the other player's actions before making a choice. These results enrich
  our understanding of LLM's social behavior and pave the way for a behavioral
  game theory for machines.
paper_ai_summary: LLMs exhibit varied behaviors in interactive games, performing well in self-interested games like the iterated Prisoner's Dilemma but sub-optimally in coordination games like the Battle of the Sexes, with behavior modifiable through additional information and prediction prompts.
paper_ai_keywords: Large Language Models (LLMs) behavioral game theory cooperation coordination finitely repeated games iterated Prisoner's Dilemma Battle of the Sexes