SOTOPIA-π: Interactive Learning of Socially Intelligent Language
  Agents
Humans learn social skills through both imitation and social interaction.
This social learning process is largely understudied by existing research on
building language agents. Motivated by this gap, we propose an interactive
learning method, SOTOPIA-pi, improving the social intelligence of language
agents. This method leverages behavior cloning and self-reinforcement training
on filtered social interaction data according to large language model (LLM)
ratings. We show that our training method allows a 7B LLM to reach the social
goal completion ability of an expert model (GPT-4-based agent), while improving
the safety of language agents and maintaining general QA ability on the MMLU
benchmark. We also find that this training paradigm uncovers some difficulties
in LLM-based evaluation of social intelligence: LLM-based evaluators
overestimate the abilities of the language agents trained specifically for
social interaction.
65f26f74d83aee7a6481db6c
Ruiyi Wang
admin_assigned
2024-03-14T08:27:30.007Z
631fec6e5ba8c026340fdda0
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Dn8NzKCYNkBiiinFEyQz5.png
Ruiyi Wang
user
Pamela153
65f26f74d83aee7a6481db6d
Haofei Yu
admin_assigned
2024-03-14T08:30:29.221Z
636453547cf2c0b4f0a3ee1e
/avatars/a453340b44d08eec2281ecbe5e993707.svg
Haofei Yu
user
lwaekfjlk
65f26f74d83aee7a6481db6e
Wenxin Zhang
admin_assigned
2024-03-14T08:30:39.910Z
62ffa14d59b9ff1ccb54b3da
/avatars/230b1a9a6f2dad96048a890ef8b97abc.svg
Wenxin Zhang
user
wzhang1992
65f26f74d83aee7a6481db6f
Zhengyang Qi
65f26f74d83aee7a6481db70
Maarten Sap
admin_assigned
2024-03-14T08:30:54.624Z
62e0657168d80187d20e7b0e
/avatars/d8de3ec3edc381b34e6f908eeeea86ee.svg
Maarten Sap
user
maartensap
65f26f74d83aee7a6481db71
Graham Neubig
admin_assigned
2024-03-14T08:31:00.237Z
60de14638bedd2315529d43f
https://cdn-avatars.huggingface.co/v1/production/uploads/1625166923504-noauth.png
Graham Neubig
user
gneubig
65f26f74d83aee7a6481db72
Yonatan Bisk
admin_assigned
2024-03-14T08:31:06.019Z
63180f3c879073cf83f135b0
https://cdn-avatars.huggingface.co/v1/production/uploads/63180f3c879073cf83f135b0/0ycXVKOywlU1hcvQ4MDws.jpeg
Yonatan Bisk
user
ybisk
65f26f74d83aee7a6481db73
Hao Zhu
claimed_verified
2024-03-14T12:31:49.629Z
61aa376688c20eebf1e8deb3
/avatars/7c11dcb232c73547d7d87834be287822.svg
Hao Zhu
user
ProKil
SOTOPIA-π: Interactive Learning of Socially Intelligent Language
  Agents
Humans learn social skills through both imitation and social interaction.
This social learning process is largely understudied by existing research on
building language agents. Motivated by this gap, we propose an interactive
learning method, SOTOPIA-pi, improving the social intelligence of language
agents. This method leverages behavior cloning and self-reinforcement training
on filtered social interaction data according to large language model (LLM)
ratings. We show that our training method allows a 7B LLM to reach the social
goal completion ability of an expert model (GPT-4-based agent), while improving
the safety of language agents and maintaining general QA ability on the MMLU
benchmark. We also find that this training paradigm uncovers some difficulties
in LLM-based evaluation of social intelligence: LLM-based evaluators
overestimate the abilities of the language agents trained specifically for
social interaction.
Interactive learning method SOTOPIA-π improves social intelligence in language agents through behavior cloning and self-reinforcement training, enhancing goal completion and safety without sacrificing general QA ability.
behavior cloning self-reinforcement training large language model (LLM) social intelligence expert model (GPT-4-based agent) safety MMLU benchmark LLM-based evaluation social interaction