title: SOTOPIA-π: Interactive Learning of Socially Intelligent Language
    Agents
summary: Humans learn social skills through both imitation and social interaction.
  This social learning process is largely understudied by existing research on
  building language agents. Motivated by this gap, we propose an interactive
  learning method, SOTOPIA-pi, improving the social intelligence of language
  agents. This method leverages behavior cloning and self-reinforcement training
  on filtered social interaction data according to large language model (LLM)
  ratings. We show that our training method allows a 7B LLM to reach the social
  goal completion ability of an expert model (GPT-4-based agent), while improving
  the safety of language agents and maintaining general QA ability on the MMLU
  benchmark. We also find that this training paradigm uncovers some difficulties
  in LLM-based evaluation of social intelligence: LLM-based evaluators
  overestimate the abilities of the language agents trained specifically for
  social interaction.
paper_authors._id: 65f26f74d83aee7a6481db6c
paper_authors.name: Ruiyi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:27:30.007Z
paper_authors._id: 631fec6e5ba8c026340fdda0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Dn8NzKCYNkBiiinFEyQz5.png
paper_authors.fullname: Ruiyi Wang
paper_authors.type: user
paper_authors.user: Pamela153
paper_authors._id: 65f26f74d83aee7a6481db6d
paper_authors.name: Haofei Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:30:29.221Z
paper_authors._id: 636453547cf2c0b4f0a3ee1e
paper_authors.avatarUrl: /avatars/a453340b44d08eec2281ecbe5e993707.svg
paper_authors.fullname: Haofei Yu
paper_authors.type: user
paper_authors.user: lwaekfjlk
paper_authors._id: 65f26f74d83aee7a6481db6e
paper_authors.name: Wenxin Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:30:39.910Z
paper_authors._id: 62ffa14d59b9ff1ccb54b3da
paper_authors.avatarUrl: /avatars/230b1a9a6f2dad96048a890ef8b97abc.svg
paper_authors.fullname: Wenxin Zhang
paper_authors.type: user
paper_authors.user: wzhang1992
paper_authors._id: 65f26f74d83aee7a6481db6f
paper_authors.name: Zhengyang Qi
paper_authors._id: 65f26f74d83aee7a6481db70
paper_authors.name: Maarten Sap
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:30:54.624Z
paper_authors._id: 62e0657168d80187d20e7b0e
paper_authors.avatarUrl: /avatars/d8de3ec3edc381b34e6f908eeeea86ee.svg
paper_authors.fullname: Maarten Sap
paper_authors.type: user
paper_authors.user: maartensap
paper_authors._id: 65f26f74d83aee7a6481db71
paper_authors.name: Graham Neubig
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:31:00.237Z
paper_authors._id: 60de14638bedd2315529d43f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1625166923504-noauth.png
paper_authors.fullname: Graham Neubig
paper_authors.type: user
paper_authors.user: gneubig
paper_authors._id: 65f26f74d83aee7a6481db72
paper_authors.name: Yonatan Bisk
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:31:06.019Z
paper_authors._id: 63180f3c879073cf83f135b0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63180f3c879073cf83f135b0/0ycXVKOywlU1hcvQ4MDws.jpeg
paper_authors.fullname: Yonatan Bisk
paper_authors.type: user
paper_authors.user: ybisk
paper_authors._id: 65f26f74d83aee7a6481db73
paper_authors.name: Hao Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-14T12:31:49.629Z
paper_authors._id: 61aa376688c20eebf1e8deb3
paper_authors.avatarUrl: /avatars/7c11dcb232c73547d7d87834be287822.svg
paper_authors.fullname: Hao Zhu
paper_authors.type: user
paper_authors.user: ProKil
paper_title: SOTOPIA-π: Interactive Learning of Socially Intelligent Language
    Agents
paper_summary: Humans learn social skills through both imitation and social interaction.
  This social learning process is largely understudied by existing research on
  building language agents. Motivated by this gap, we propose an interactive
  learning method, SOTOPIA-pi, improving the social intelligence of language
  agents. This method leverages behavior cloning and self-reinforcement training
  on filtered social interaction data according to large language model (LLM)
  ratings. We show that our training method allows a 7B LLM to reach the social
  goal completion ability of an expert model (GPT-4-based agent), while improving
  the safety of language agents and maintaining general QA ability on the MMLU
  benchmark. We also find that this training paradigm uncovers some difficulties
  in LLM-based evaluation of social intelligence: LLM-based evaluators
  overestimate the abilities of the language agents trained specifically for
  social interaction.
paper_ai_summary: Interactive learning method SOTOPIA-π improves social intelligence in language agents through behavior cloning and self-reinforcement training, enhancing goal completion and safety without sacrificing general QA ability.
paper_ai_keywords: behavior cloning self-reinforcement training large language model (LLM) social intelligence expert model (GPT-4-based agent) safety MMLU benchmark LLM-based evaluation social interaction