title: CoSER: Coordinating LLM-Based Persona Simulation of Established Roles
summary: Role-playing language agents (RPLAs) have emerged as promising applications
  of large language models (LLMs). However, simulating established characters
  presents a challenging task for RPLAs, due to the lack of authentic character
  datasets and nuanced evaluation methods using such data. In this paper, we
  present CoSER, a collection of a high-quality dataset, open models, and an
  evaluation protocol towards effective RPLAs of established characters. The
  CoSER dataset covers 17,966 characters from 771 renowned books. It provides
  authentic dialogues with real-world intricacies, as well as diverse data types
  such as conversation setups, character experiences and internal thoughts.
  Drawing from acting methodology, we introduce given-circumstance acting for
  training and evaluating role-playing LLMs, where LLMs sequentially portray
  multiple characters in book scenes. Using our dataset, we develop CoSER 8B and
  CoSER 70B, i.e., advanced open role-playing LLMs built on LLaMA-3.1 models.
  Extensive experiments demonstrate the value of the CoSER dataset for RPLA
  training, evaluation and retrieval. Moreover, CoSER 70B exhibits
  state-of-the-art performance surpassing or matching GPT-4o on our evaluation
  and three existing benchmarks, i.e., achieving 75.80% and 93.47% accuracy on
  the InCharacter and LifeChoice benchmarks respectively.
paper_authors._id: 67aee90c208d299238758622
paper_authors.name: Xintao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T14:37:56.459Z
paper_authors._id: 64c7bf2c4524c2aea7eac0b3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c7bf2c4524c2aea7eac0b3/5ocZ69MvN4RFv86Aa7ks3.png
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Neph0s
paper_authors._id: 67aee90c208d299238758623
paper_authors.name: Heng Wang
paper_authors._id: 67aee90c208d299238758624
paper_authors.name: Yifei Zhang
paper_authors._id: 67aee90c208d299238758625
paper_authors.name: Xinfeng Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T09:37:50.630Z
paper_authors._id: 6749b9b54431ba7184411328
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/c2DvvGF_Ga5rKY9iJuyib.png
paper_authors.fullname: Xinfeng
paper_authors.type: user
paper_authors.user: Joanna-Yuan
paper_authors._id: 67aee90c208d299238758626
paper_authors.name: Rui Xu
paper_authors._id: 67aee90c208d299238758627
paper_authors.name: Jen-tse Huang
paper_authors._id: 67aee90c208d299238758628
paper_authors.name: Siyu Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:43:00.148Z
paper_authors._id: 62d62b333bf5e059f7d2b286
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668513815771-62d62b333bf5e059f7d2b286.jpeg
paper_authors.fullname: Siyu Yuan
paper_authors.type: user
paper_authors.user: siyuyuan
paper_authors._id: 67aee90c208d299238758629
paper_authors.name: Haoran Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:42:42.152Z
paper_authors._id: 64f06b030ba7caa06a8bf13e
paper_authors.avatarUrl: /avatars/a9dce43ecdd0339c438a64e28dcd3fcf.svg
paper_authors.fullname: Haoran Guo
paper_authors.type: user
paper_authors.user: Haoran-Guo
paper_authors._id: 67aee90c208d29923875862a
paper_authors.name: Jiangjie Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:42:33.652Z
paper_authors._id: 606ed1884ffe81d1e03e81e5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1639375346654-606ed1884ffe81d1e03e81e5.png
paper_authors.fullname: Jiangjie Chen
paper_authors.type: user
paper_authors.user: jiangjiechen
paper_authors._id: 67aee90c208d29923875862b
paper_authors.name: Wei Wang
paper_authors._id: 67aee90c208d29923875862c
paper_authors.name: Yanghua Xiao
paper_authors._id: 67aee90c208d29923875862d
paper_authors.name: Shuchang Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:42:04.561Z
paper_authors._id: 620531f8522e40b4a18d872f
paper_authors.avatarUrl: /avatars/cd2fba21c499e27dea75e571a3b75228.svg
paper_authors.fullname: Shuchang Zhou
paper_authors.type: user
paper_authors.user: zsc
paper_title: CoSER: Coordinating LLM-Based Persona Simulation of Established Roles
paper_summary: Role-playing language agents (RPLAs) have emerged as promising applications
  of large language models (LLMs). However, simulating established characters
  presents a challenging task for RPLAs, due to the lack of authentic character
  datasets and nuanced evaluation methods using such data. In this paper, we
  present CoSER, a collection of a high-quality dataset, open models, and an
  evaluation protocol towards effective RPLAs of established characters. The
  CoSER dataset covers 17,966 characters from 771 renowned books. It provides
  authentic dialogues with real-world intricacies, as well as diverse data types
  such as conversation setups, character experiences and internal thoughts.
  Drawing from acting methodology, we introduce given-circumstance acting for
  training and evaluating role-playing LLMs, where LLMs sequentially portray
  multiple characters in book scenes. Using our dataset, we develop CoSER 8B and
  CoSER 70B, i.e., advanced open role-playing LLMs built on LLaMA-3.1 models.
  Extensive experiments demonstrate the value of the CoSER dataset for RPLA
  training, evaluation and retrieval. Moreover, CoSER 70B exhibits
  state-of-the-art performance surpassing or matching GPT-4o on our evaluation
  and three existing benchmarks, i.e., achieving 75.80% and 93.47% accuracy on
  the InCharacter and LifeChoice benchmarks respectively.
paper_ai_summary: CoSER, a dataset and evaluation protocol for role-playing language agents, enhances LLMs by simulating established characters with high-quality data and advanced models.
paper_ai_keywords: role-playing language agents large language models authentic character datasets nuanced evaluation methods CoSER given-circumstance acting conversation setups LLaMA-3.1 InCharacter LifeChoice