title: R3: Robust Rubric-Agnostic Reward Models
summary: Reward models are essential for aligning language model outputs with human
  preferences, yet existing approaches often lack both controllability and
  interpretability. These models are typically optimized for narrow objectives,
  limiting their generalizability to broader downstream tasks. Moreover, their
  scalar outputs are difficult to interpret without contextual reasoning. To
  address these limitations, we introduce R3, a novel reward modeling framework
  that is rubric-agnostic, generalizable across evaluation dimensions, and
  provides interpretable, reasoned score assignments. R3 enables more transparent
  and flexible evaluation of language models, supporting robust alignment with
  diverse human values and use cases. Our models, data, and code are available as
  open source at https://github.com/rubricreward/r3
paper_authors._id: 682cb3213d53f7c2576ac370
paper_authors.name: David Anugraha
paper_authors._id: 682cb3213d53f7c2576ac371
paper_authors.name: Zilu Tang
paper_authors._id: 682cb3213d53f7c2576ac372
paper_authors.name: Lester James V. Miranda
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:01:27.643Z
paper_authors._id: 634e20a0c1ce28f1de920cc4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1666064515342-noauth.jpeg
paper_authors.fullname: Lj V. Miranda
paper_authors.type: user
paper_authors.user: ljvmiranda921
paper_authors._id: 682cb3213d53f7c2576ac373
paper_authors.name: Hanyang Zhao
paper_authors._id: 682cb3213d53f7c2576ac374
paper_authors.name: Mohammad Rifqi Farhansyah
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T09:03:17.723Z
paper_authors._id: 65af40481216d503271e7d95
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65af40481216d503271e7d95/7BvAFiSHbL4rShMzdsLyd.png
paper_authors.fullname: Mohammad Rifqi Farhansyah
paper_authors.type: user
paper_authors.user: rifqifarhansyah
paper_authors._id: 682cb3213d53f7c2576ac375
paper_authors.name: Garry Kuwanto
paper_authors._id: 682cb3213d53f7c2576ac376
paper_authors.name: Derry Wijaya
paper_authors._id: 682cb3213d53f7c2576ac377
paper_authors.name: Genta Indra Winata
paper_title: R3: Robust Rubric-Agnostic Reward Models
paper_summary: Reward models are essential for aligning language model outputs with human
  preferences, yet existing approaches often lack both controllability and
  interpretability. These models are typically optimized for narrow objectives,
  limiting their generalizability to broader downstream tasks. Moreover, their
  scalar outputs are difficult to interpret without contextual reasoning. To
  address these limitations, we introduce R3, a novel reward modeling framework
  that is rubric-agnostic, generalizable across evaluation dimensions, and
  provides interpretable, reasoned score assignments. R3 enables more transparent
  and flexible evaluation of language models, supporting robust alignment with
  diverse human values and use cases. Our models, data, and code are available as
  open source at https://github.com/rubricreward/r3
paper_ai_summary: R3 is a novel reward modeling framework that enhances controllability, interpretability, and flexibility in aligning language models with human preferences.
paper_ai_keywords: reward models alignment human preferences controllability interpretability generalizability rubric-agnostic evaluation dimensions reasoned score assignments transparent evaluation