title: Training Language Models for Social Deduction with Multi-Agent
    Reinforcement Learning
summary: Communicating in natural language is a powerful tool in multi-agent settings,
  as it enables independent agents to share information in partially observable
  settings and allows zero-shot coordination with humans. However, most prior
  works are limited as they either rely on training with large amounts of human
  demonstrations or lack the ability to generate natural and useful communication
  strategies. In this work, we train language models to have productive
  discussions about their environment in natural language without any human
  demonstrations. We decompose the communication problem into listening and
  speaking. Our key idea is to leverage the agent's goal to predict useful
  information about the world as a dense reward signal that guides communication.
  Specifically, we improve a model's listening skills by training them to predict
  information about the environment based on discussions, and we simultaneously
  improve a model's speaking skills with multi-agent reinforcement learning by
  rewarding messages based on their influence on other agents. To investigate the
  role and necessity of communication in complex social settings, we study an
  embodied social deduction game based on Among Us, where the key question to
  answer is the identity of an adversarial imposter. We analyze emergent
  behaviors due to our technique, such as accusing suspects and providing
  evidence, and find that it enables strong discussions, doubling the win rates
  compared to standard RL. We release our code and models at
  https://socialdeductionllm.github.io/
paper_authors._id: 67ab1314385da1f07cda1271
paper_authors.name: Bidipta Sarkar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T09:51:17.933Z
paper_authors._id: 63abbf74ad514ca8d14a0548
paper_authors.avatarUrl: /avatars/b1357b73b8f9a8ff9908710ad64154ef.svg
paper_authors.fullname: Bidipta Sarkar
paper_authors.type: user
paper_authors.user: bidiptas
paper_authors._id: 67ab1314385da1f07cda1272
paper_authors.name: Warren Xia
paper_authors._id: 67ab1314385da1f07cda1273
paper_authors.name: C. Karen Liu
paper_authors._id: 67ab1314385da1f07cda1274
paper_authors.name: Dorsa Sadigh
paper_title: Training Language Models for Social Deduction with Multi-Agent
    Reinforcement Learning
paper_summary: Communicating in natural language is a powerful tool in multi-agent settings,
  as it enables independent agents to share information in partially observable
  settings and allows zero-shot coordination with humans. However, most prior
  works are limited as they either rely on training with large amounts of human
  demonstrations or lack the ability to generate natural and useful communication
  strategies. In this work, we train language models to have productive
  discussions about their environment in natural language without any human
  demonstrations. We decompose the communication problem into listening and
  speaking. Our key idea is to leverage the agent's goal to predict useful
  information about the world as a dense reward signal that guides communication.
  Specifically, we improve a model's listening skills by training them to predict
  information about the environment based on discussions, and we simultaneously
  improve a model's speaking skills with multi-agent reinforcement learning by
  rewarding messages based on their influence on other agents. To investigate the
  role and necessity of communication in complex social settings, we study an
  embodied social deduction game based on Among Us, where the key question to
  answer is the identity of an adversarial imposter. We analyze emergent
  behaviors due to our technique, such as accusing suspects and providing
  evidence, and find that it enables strong discussions, doubling the win rates
  compared to standard RL. We release our code and models at
  https://socialdeductionllm.github.io/
paper_ai_summary: Language models trained without human demonstrations improve multi-agent communication skills through reinforcement learning, enabling effective zero-shot coordination in complex social deduction tasks.
paper_ai_keywords: language models multi-agent reinforcement learning communication strategies dense reward signal embodiment social deduction adversarial imposter emergent behaviors zero-shot coordination