ChARM: Character-based Act-adaptive Reward Modeling for Advanced
  Role-Playing Language Agents
Role-Playing Language Agents (RPLAs) aim to simulate characters for realistic
and engaging human-computer interactions. However, traditional reward models
often struggle with scalability and adapting to subjective conversational
preferences. We propose ChARM, a Character-based Act-adaptive Reward Model,
addressing these challenges through two innovations: (1) an act-adaptive margin
that significantly enhances learning efficiency and generalizability, and (2) a
self-evolution mechanism leveraging large-scale unlabeled data to improve
training coverage. Additionally, we introduce RoleplayPref, the first
large-scale preference dataset specifically for RPLAs, featuring 1,108
characters, 13 subcategories, and 16,888 bilingual dialogues, alongside
RoleplayEval, a dedicated evaluation benchmark. Experimental results show a 13%
improvement over the conventional Bradley-Terry model in preference rankings.
Furthermore, applying ChARM-generated rewards to preference learning techniques
(e.g., direct preference optimization) achieves state-of-the-art results on
CharacterEval and RoleplayEval. Code and dataset are available at
https://github.com/calubkk/ChARM.
683dd20fb1080514bd945374
Feiteng Fang
claimed_verified
2025-06-10T09:29:29.714Z
634d4284f0a69955f66489f3
/avatars/9fac134537e6ef802f26d6d54c0e0bca.svg
feitengfang
user
feltoner
683dd20fb1080514bd945375
Ting-En Lin
683dd20fb1080514bd945376
Yuchuan Wu
683dd20fb1080514bd945377
Xiong Liu
683dd20fb1080514bd945378
Xiang Huang
683dd20fb1080514bd945379
Dingwei Chen
683dd20fb1080514bd94537a
Jing Ye
683dd20fb1080514bd94537b
Haonan Zhang
683dd20fb1080514bd94537c
Liang Zhu
683dd20fb1080514bd94537d
Hamid Alinejad-Rokny
683dd20fb1080514bd94537e
Min Yang
683dd20fb1080514bd94537f
Fei Huang
683dd20fb1080514bd945380
Yongbin Li
ChARM: Character-based Act-adaptive Reward Modeling for Advanced
  Role-Playing Language Agents
Role-Playing Language Agents (RPLAs) aim to simulate characters for realistic
and engaging human-computer interactions. However, traditional reward models
often struggle with scalability and adapting to subjective conversational
preferences. We propose ChARM, a Character-based Act-adaptive Reward Model,
addressing these challenges through two innovations: (1) an act-adaptive margin
that significantly enhances learning efficiency and generalizability, and (2) a
self-evolution mechanism leveraging large-scale unlabeled data to improve
training coverage. Additionally, we introduce RoleplayPref, the first
large-scale preference dataset specifically for RPLAs, featuring 1,108
characters, 13 subcategories, and 16,888 bilingual dialogues, alongside
RoleplayEval, a dedicated evaluation benchmark. Experimental results show a 13%
improvement over the conventional Bradley-Terry model in preference rankings.
Furthermore, applying ChARM-generated rewards to preference learning techniques
(e.g., direct preference optimization) achieves state-of-the-art results on
CharacterEval and RoleplayEval. Code and dataset are available at
https://github.com/calubkk/ChARM.
ChARM, a character-focused adaptive reward model, improves preference learning for role-playing language agents by using an act-adaptive margin and self-evolution with unlabeled data, achieving superior results on dedicated benchmarks.
Role-Playing Language Agents ChARM act-adaptive margin self-evolution mechanism RoleplayPref RoleplayEval preference learning direct preference optimization CharacterEval