The Consensus Game: Language Model Generation via Equilibrium Search
When applied to question answering and other text generation tasks, language
models (LMs) may be queried generatively (by sampling answers from their output
distribution) or discriminatively (by using them to score or rank a set of
candidate outputs). These procedures sometimes yield very different
predictions. How do we reconcile mutually incompatible scoring procedures to
obtain coherent LM predictions? We introduce a new, a training-free,
game-theoretic procedure for language model decoding. Our approach casts
language model decoding as a regularized imperfect-information sequential
signaling game - which we term the CONSENSUS GAME - in which a GENERATOR seeks
to communicate an abstract correctness parameter using natural language
sentences to a DISCRIMINATOR. We develop computational procedures for finding
approximate equilibria of this game, resulting in a decoding algorithm we call
EQUILIBRIUM-RANKING. Applied to a large number of tasks (including reading
comprehension, commonsense reasoning, mathematical problem-solving, and
dialog), EQUILIBRIUM-RANKING consistently, and sometimes substantially,
improves performance over existing LM decoding procedures - on multiple
benchmarks, we observe that applying EQUILIBRIUM-RANKING to LLaMA-7B
outperforms the much larger LLaMA-65B and PaLM-540B models. These results
highlight the promise of game-theoretic tools for addressing fundamental
challenges of truthfulness and consistency in LMs.
652cc66c66313ebb61c91e03
Athul Paul Jacob
claimed_verified
2023-10-16T12:05:55.246Z
652d266336986a014f87e43b
/avatars/46c9de75ac6d2af3da6aa4cbf870adaf.svg
Athul Paul Jacob
user
apjacob
652cc66c66313ebb61c91e04
Yikang Shen
admin_assigned
2023-10-16T08:22:28.350Z
6463995939359568c63d740c
/avatars/11caacefc68c1259dbc8764cc4340481.svg
Yikang Shen
user
YikangS
652cc66c66313ebb61c91e05
Gabriele Farina
claimed_verified
2025-03-24T08:57:17.817Z
67dc549ac2546c44dfaf10bd
/avatars/3ddb42be2c8e8f97b6905b4a111e0b29.svg
Gabriele Farina
user
gfarina
652cc66c66313ebb61c91e06
Jacob Andreas
The Consensus Game: Language Model Generation via Equilibrium Search
When applied to question answering and other text generation tasks, language
models (LMs) may be queried generatively (by sampling answers from their output
distribution) or discriminatively (by using them to score or rank a set of
candidate outputs). These procedures sometimes yield very different
predictions. How do we reconcile mutually incompatible scoring procedures to
obtain coherent LM predictions? We introduce a new, a training-free,
game-theoretic procedure for language model decoding. Our approach casts
language model decoding as a regularized imperfect-information sequential
signaling game - which we term the CONSENSUS GAME - in which a GENERATOR seeks
to communicate an abstract correctness parameter using natural language
sentences to a DISCRIMINATOR. We develop computational procedures for finding
approximate equilibria of this game, resulting in a decoding algorithm we call
EQUILIBRIUM-RANKING. Applied to a large number of tasks (including reading
comprehension, commonsense reasoning, mathematical problem-solving, and
dialog), EQUILIBRIUM-RANKING consistently, and sometimes substantially,
improves performance over existing LM decoding procedures - on multiple
benchmarks, we observe that applying EQUILIBRIUM-RANKING to LLaMA-7B
outperforms the much larger LLaMA-65B and PaLM-540B models. These results
highlight the promise of game-theoretic tools for addressing fundamental
challenges of truthfulness and consistency in LMs.
A new game-theoretic decoding method, EQUILIBRIUM-RANKING, improves performance across various text generation tasks by reconciling incompatible scoring procedures.
language models generative discriminative game-theoretic CONSENSUS GAME GENERATOR DISCRIMINATOR equilibrium EQUILIBRIUM-RANKING reading comprehension commonsense reasoning mathematical problem-solving dialog LLaMA-7B LLaMA-65B PaLM-540B truthfulness consistency