title: PRMBench: A Fine-grained and Challenging Benchmark for Process-Level
    Reward Models
summary: Process-level Reward Models (PRMs) are crucial for complex reasoning and
  decision-making tasks, where each intermediate step plays an important role in
  the reasoning process. Since language models are prone to various types of
  errors during the reasoning process, PRMs are required to possess nuanced
  capabilities for detecting various implicit error types in real-world
  scenarios. However, current benchmarks primarily focus on step correctness,
  failing to evaluate PRMs' performance systematically. To address this gap, we
  introduce PRMBench, a process-level benchmark specifically designed to assess
  the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216
  carefully designed problems and 83,456 step-level labels, evaluating models
  across multiple dimensions, including simplicity, soundness, and sensitivity.
  In our experiments on 15 models, spanning both open-source PRMs and
  closed-source large language models prompted as critic models, we uncover
  significant weaknesses in current PRMs. These findings underscore the
  challenges inherent in process-level evaluation and highlight key directions
  for future research. We hope PRMBench can be a robust bench for advancing
  research on PRM evaluation and development.
paper_authors._id: 677ccbcffc5146d9bc9b5d73
paper_authors.name: Mingyang Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-07T13:01:57.016Z
paper_authors._id: 66aca01e33f6b27979856f6f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66aca01e33f6b27979856f6f/IyOxv89TudwscGH7tdue3.jpeg
paper_authors.fullname: Mingyang Song
paper_authors.type: user
paper_authors.user: hitsmy
paper_authors._id: 677ccbcffc5146d9bc9b5d74
paper_authors.name: Zhaochen Su
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-10T08:04:28.602Z
paper_authors._id: 64264095ba51f8a2136946a0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64264095ba51f8a2136946a0/FR33boVpkDXcrvGMBmprF.jpeg
paper_authors.fullname: Zhaochen Su
paper_authors.type: user
paper_authors.user: Warrieryes
paper_authors._id: 677ccbcffc5146d9bc9b5d75
paper_authors.name: Xiaoye Qu
paper_authors._id: 677ccbcffc5146d9bc9b5d76
paper_authors.name: Jiawei Zhou
paper_authors._id: 677ccbcffc5146d9bc9b5d77
paper_authors.name: Yu Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-23T15:05:15.015Z
paper_authors._id: 67017abfe4d49b157ac534d9
paper_authors.avatarUrl: /avatars/997e1b9f54b27a7728a9d4abfee4ba91.svg
paper_authors.fullname: Yu Cheng
paper_authors.type: user
paper_authors.user: ych133
paper_title: PRMBench: A Fine-grained and Challenging Benchmark for Process-Level
    Reward Models
paper_summary: Process-level Reward Models (PRMs) are crucial for complex reasoning and
  decision-making tasks, where each intermediate step plays an important role in
  the reasoning process. Since language models are prone to various types of
  errors during the reasoning process, PRMs are required to possess nuanced
  capabilities for detecting various implicit error types in real-world
  scenarios. However, current benchmarks primarily focus on step correctness,
  failing to evaluate PRMs' performance systematically. To address this gap, we
  introduce PRMBench, a process-level benchmark specifically designed to assess
  the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216
  carefully designed problems and 83,456 step-level labels, evaluating models
  across multiple dimensions, including simplicity, soundness, and sensitivity.
  In our experiments on 15 models, spanning both open-source PRMs and
  closed-source large language models prompted as critic models, we uncover
  significant weaknesses in current PRMs. These findings underscore the
  challenges inherent in process-level evaluation and highlight key directions
  for future research. We hope PRMBench can be a robust bench for advancing
  research on PRM evaluation and development.