CoSER: Coordinating LLM-Based Persona Simulation of Established Roles
Role-playing language agents (RPLAs) have emerged as promising applications
of large language models (LLMs). However, simulating established characters
presents a challenging task for RPLAs, due to the lack of authentic character
datasets and nuanced evaluation methods using such data. In this paper, we
present CoSER, a collection of a high-quality dataset, open models, and an
evaluation protocol towards effective RPLAs of established characters. The
CoSER dataset covers 17,966 characters from 771 renowned books. It provides
authentic dialogues with real-world intricacies, as well as diverse data types
such as conversation setups, character experiences and internal thoughts.
Drawing from acting methodology, we introduce given-circumstance acting for
training and evaluating role-playing LLMs, where LLMs sequentially portray
multiple characters in book scenes. Using our dataset, we develop CoSER 8B and
CoSER 70B, i.e., advanced open role-playing LLMs built on LLaMA-3.1 models.
Extensive experiments demonstrate the value of the CoSER dataset for RPLA
training, evaluation and retrieval. Moreover, CoSER 70B exhibits
state-of-the-art performance surpassing or matching GPT-4o on our evaluation
and three existing benchmarks, i.e., achieving 75.80% and 93.47% accuracy on
the InCharacter and LifeChoice benchmarks respectively.
67aee90c208d299238758622
Xintao Wang
claimed_verified
2025-02-19T14:37:56.459Z
64c7bf2c4524c2aea7eac0b3
https://cdn-avatars.huggingface.co/v1/production/uploads/64c7bf2c4524c2aea7eac0b3/5ocZ69MvN4RFv86Aa7ks3.png
Xintao Wang
user
Neph0s
67aee90c208d299238758623
Heng Wang
67aee90c208d299238758624
Yifei Zhang
67aee90c208d299238758625
Xinfeng Yuan
claimed_verified
2025-02-20T09:37:50.630Z
6749b9b54431ba7184411328
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/c2DvvGF_Ga5rKY9iJuyib.png
Xinfeng
user
Joanna-Yuan
67aee90c208d299238758626
Rui Xu
67aee90c208d299238758627
Jen-tse Huang
67aee90c208d299238758628
Siyu Yuan
admin_assigned
2025-02-14T12:43:00.148Z
62d62b333bf5e059f7d2b286
https://cdn-avatars.huggingface.co/v1/production/uploads/1668513815771-62d62b333bf5e059f7d2b286.jpeg
Siyu Yuan
user
siyuyuan
67aee90c208d299238758629
Haoran Guo
admin_assigned
2025-02-14T12:42:42.152Z
64f06b030ba7caa06a8bf13e
/avatars/a9dce43ecdd0339c438a64e28dcd3fcf.svg
Haoran Guo
user
Haoran-Guo
67aee90c208d29923875862a
Jiangjie Chen
admin_assigned
2025-02-14T12:42:33.652Z
606ed1884ffe81d1e03e81e5
https://cdn-avatars.huggingface.co/v1/production/uploads/1639375346654-606ed1884ffe81d1e03e81e5.png
Jiangjie Chen
user
jiangjiechen
67aee90c208d29923875862b
Wei Wang
67aee90c208d29923875862c
Yanghua Xiao
67aee90c208d29923875862d
Shuchang Zhou
admin_assigned
2025-02-14T12:42:04.561Z
620531f8522e40b4a18d872f
/avatars/cd2fba21c499e27dea75e571a3b75228.svg
Shuchang Zhou
user
zsc
CoSER: Coordinating LLM-Based Persona Simulation of Established Roles
Role-playing language agents (RPLAs) have emerged as promising applications
of large language models (LLMs). However, simulating established characters
presents a challenging task for RPLAs, due to the lack of authentic character
datasets and nuanced evaluation methods using such data. In this paper, we
present CoSER, a collection of a high-quality dataset, open models, and an
evaluation protocol towards effective RPLAs of established characters. The
CoSER dataset covers 17,966 characters from 771 renowned books. It provides
authentic dialogues with real-world intricacies, as well as diverse data types
such as conversation setups, character experiences and internal thoughts.
Drawing from acting methodology, we introduce given-circumstance acting for
training and evaluating role-playing LLMs, where LLMs sequentially portray
multiple characters in book scenes. Using our dataset, we develop CoSER 8B and
CoSER 70B, i.e., advanced open role-playing LLMs built on LLaMA-3.1 models.
Extensive experiments demonstrate the value of the CoSER dataset for RPLA
training, evaluation and retrieval. Moreover, CoSER 70B exhibits
state-of-the-art performance surpassing or matching GPT-4o on our evaluation
and three existing benchmarks, i.e., achieving 75.80% and 93.47% accuracy on
the InCharacter and LifeChoice benchmarks respectively.
CoSER, a dataset and evaluation protocol for role-playing language agents, enhances LLMs by simulating established characters with high-quality data and advanced models.
role-playing language agents large language models authentic character datasets nuanced evaluation methods CoSER given-circumstance acting conversation setups LLaMA-3.1 InCharacter LifeChoice