title: The Consensus Game: Language Model Generation via Equilibrium Search
summary: When applied to question answering and other text generation tasks, language
  models (LMs) may be queried generatively (by sampling answers from their output
  distribution) or discriminatively (by using them to score or rank a set of
  candidate outputs). These procedures sometimes yield very different
  predictions. How do we reconcile mutually incompatible scoring procedures to
  obtain coherent LM predictions? We introduce a new, a training-free,
  game-theoretic procedure for language model decoding. Our approach casts
  language model decoding as a regularized imperfect-information sequential
  signaling game - which we term the CONSENSUS GAME - in which a GENERATOR seeks
  to communicate an abstract correctness parameter using natural language
  sentences to a DISCRIMINATOR. We develop computational procedures for finding
  approximate equilibria of this game, resulting in a decoding algorithm we call
  EQUILIBRIUM-RANKING. Applied to a large number of tasks (including reading
  comprehension, commonsense reasoning, mathematical problem-solving, and
  dialog), EQUILIBRIUM-RANKING consistently, and sometimes substantially,
  improves performance over existing LM decoding procedures - on multiple
  benchmarks, we observe that applying EQUILIBRIUM-RANKING to LLaMA-7B
  outperforms the much larger LLaMA-65B and PaLM-540B models. These results
  highlight the promise of game-theoretic tools for addressing fundamental
  challenges of truthfulness and consistency in LMs.
paper_authors._id: 652cc66c66313ebb61c91e03
paper_authors.name: Athul Paul Jacob
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-16T12:05:55.246Z
paper_authors._id: 652d266336986a014f87e43b
paper_authors.avatarUrl: /avatars/46c9de75ac6d2af3da6aa4cbf870adaf.svg
paper_authors.fullname: Athul Paul Jacob
paper_authors.type: user
paper_authors.user: apjacob
paper_authors._id: 652cc66c66313ebb61c91e04
paper_authors.name: Yikang Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-16T08:22:28.350Z
paper_authors._id: 6463995939359568c63d740c
paper_authors.avatarUrl: /avatars/11caacefc68c1259dbc8764cc4340481.svg
paper_authors.fullname: Yikang Shen
paper_authors.type: user
paper_authors.user: YikangS
paper_authors._id: 652cc66c66313ebb61c91e05
paper_authors.name: Gabriele Farina
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-24T08:57:17.817Z
paper_authors._id: 67dc549ac2546c44dfaf10bd
paper_authors.avatarUrl: /avatars/3ddb42be2c8e8f97b6905b4a111e0b29.svg
paper_authors.fullname: Gabriele Farina
paper_authors.type: user
paper_authors.user: gfarina
paper_authors._id: 652cc66c66313ebb61c91e06
paper_authors.name: Jacob Andreas
paper_title: The Consensus Game: Language Model Generation via Equilibrium Search
paper_summary: When applied to question answering and other text generation tasks, language
  models (LMs) may be queried generatively (by sampling answers from their output
  distribution) or discriminatively (by using them to score or rank a set of
  candidate outputs). These procedures sometimes yield very different
  predictions. How do we reconcile mutually incompatible scoring procedures to
  obtain coherent LM predictions? We introduce a new, a training-free,
  game-theoretic procedure for language model decoding. Our approach casts
  language model decoding as a regularized imperfect-information sequential
  signaling game - which we term the CONSENSUS GAME - in which a GENERATOR seeks
  to communicate an abstract correctness parameter using natural language
  sentences to a DISCRIMINATOR. We develop computational procedures for finding
  approximate equilibria of this game, resulting in a decoding algorithm we call
  EQUILIBRIUM-RANKING. Applied to a large number of tasks (including reading
  comprehension, commonsense reasoning, mathematical problem-solving, and
  dialog), EQUILIBRIUM-RANKING consistently, and sometimes substantially,
  improves performance over existing LM decoding procedures - on multiple
  benchmarks, we observe that applying EQUILIBRIUM-RANKING to LLaMA-7B
  outperforms the much larger LLaMA-65B and PaLM-540B models. These results
  highlight the promise of game-theoretic tools for addressing fundamental
  challenges of truthfulness and consistency in LMs.
paper_ai_summary: A new game-theoretic decoding method, EQUILIBRIUM-RANKING, improves performance across various text generation tasks by reconciling incompatible scoring procedures.
paper_ai_keywords: language models generative discriminative game-theoretic CONSENSUS GAME GENERATOR DISCRIMINATOR equilibrium EQUILIBRIUM-RANKING reading comprehension commonsense reasoning mathematical problem-solving dialog LLaMA-7B LLaMA-65B PaLM-540B truthfulness consistency