title: Role-Playing Evaluation for Large Language Models
summary: Large Language Models (LLMs) demonstrate a notable capacity for adopting
  personas and engaging in role-playing. However, evaluating this ability
  presents significant challenges, as human assessments are resource-intensive
  and automated evaluations can be biased. To address this, we introduce
  Role-Playing Eval (RPEval), a novel benchmark designed to assess LLM
  role-playing capabilities across four key dimensions: emotional understanding,
  decision-making, moral alignment, and in-character consistency. This article
  details the construction of RPEval and presents baseline evaluations. Our code
  and dataset are available at https://github.com/yelboudouri/RPEval
paper_authors._id: 683b58eb84fbd4b28d8d891e
paper_authors.name: Yassine El Boudouri
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:42:17.658Z
paper_authors._id: 6469408ab2321e47d3294414
paper_authors.avatarUrl: /avatars/05621d33f9f6337bc66e59d9e81d05ef.svg
paper_authors.fullname: Yassine El Boudouri
paper_authors.type: user
paper_authors.user: yelboudouri
paper_authors._id: 683b58eb84fbd4b28d8d891f
paper_authors.name: Walter Nuninger
paper_authors._id: 683b58eb84fbd4b28d8d8920
paper_authors.name: Julian Alvarez
paper_authors._id: 683b58eb84fbd4b28d8d8921
paper_authors.name: Yvan Peter
paper_title: Role-Playing Evaluation for Large Language Models
paper_summary: Large Language Models (LLMs) demonstrate a notable capacity for adopting
  personas and engaging in role-playing. However, evaluating this ability
  presents significant challenges, as human assessments are resource-intensive
  and automated evaluations can be biased. To address this, we introduce
  Role-Playing Eval (RPEval), a novel benchmark designed to assess LLM
  role-playing capabilities across four key dimensions: emotional understanding,
  decision-making, moral alignment, and in-character consistency. This article
  details the construction of RPEval and presents baseline evaluations. Our code
  and dataset are available at https://github.com/yelboudouri/RPEval
paper_ai_summary: A benchmark called Role-Playing Eval assesses Large Language Models in role-playing across emotional understanding, decision-making, moral alignment, and in-character consistency.
paper_ai_keywords: Large Language Models Role-Playing Eval emotional understanding decision-making moral alignment in-character consistency