PRMBench: A Fine-grained and Challenging Benchmark for Process-Level
  Reward Models
Process-level Reward Models (PRMs) are crucial for complex reasoning and
decision-making tasks, where each intermediate step plays an important role in
the reasoning process. Since language models are prone to various types of
errors during the reasoning process, PRMs are required to possess nuanced
capabilities for detecting various implicit error types in real-world
scenarios. However, current benchmarks primarily focus on step correctness,
failing to evaluate PRMs' performance systematically. To address this gap, we
introduce PRMBench, a process-level benchmark specifically designed to assess
the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216
carefully designed problems and 83,456 step-level labels, evaluating models
across multiple dimensions, including simplicity, soundness, and sensitivity.
In our experiments on 15 models, spanning both open-source PRMs and
closed-source large language models prompted as critic models, we uncover
significant weaknesses in current PRMs. These findings underscore the
challenges inherent in process-level evaluation and highlight key directions
for future research. We hope PRMBench can be a robust bench for advancing
research on PRM evaluation and development.
677ccbcffc5146d9bc9b5d73
Mingyang Song
claimed_verified
2025-01-07T13:01:57.016Z
66aca01e33f6b27979856f6f
https://cdn-avatars.huggingface.co/v1/production/uploads/66aca01e33f6b27979856f6f/IyOxv89TudwscGH7tdue3.jpeg
Mingyang Song
user
hitsmy
677ccbcffc5146d9bc9b5d74
Zhaochen Su
extracted_confirmed
2025-01-10T08:04:28.602Z
64264095ba51f8a2136946a0
https://cdn-avatars.huggingface.co/v1/production/uploads/64264095ba51f8a2136946a0/FR33boVpkDXcrvGMBmprF.jpeg
Zhaochen Su
user
Warrieryes
677ccbcffc5146d9bc9b5d75
Xiaoye Qu
677ccbcffc5146d9bc9b5d76
Jiawei Zhou
677ccbcffc5146d9bc9b5d77
Yu Cheng
claimed_verified
2025-01-23T15:05:15.015Z
67017abfe4d49b157ac534d9
/avatars/997e1b9f54b27a7728a9d4abfee4ba91.svg
Yu Cheng
user
ych133
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level
  Reward Models
Process-level Reward Models (PRMs) are crucial for complex reasoning and
decision-making tasks, where each intermediate step plays an important role in
the reasoning process. Since language models are prone to various types of
errors during the reasoning process, PRMs are required to possess nuanced
capabilities for detecting various implicit error types in real-world
scenarios. However, current benchmarks primarily focus on step correctness,
failing to evaluate PRMs' performance systematically. To address this gap, we
introduce PRMBench, a process-level benchmark specifically designed to assess
the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216
carefully designed problems and 83,456 step-level labels, evaluating models
across multiple dimensions, including simplicity, soundness, and sensitivity.
In our experiments on 15 models, spanning both open-source PRMs and
closed-source large language models prompted as critic models, we uncover
significant weaknesses in current PRMs. These findings underscore the
challenges inherent in process-level evaluation and highlight key directions
for future research. We hope PRMBench can be a robust bench for advancing
research on PRM evaluation and development.
PRMBench is a process-level benchmark designed to evaluate the fine-grained error detection capabilities of Process-Level Reward Models (PRMs) across multiple dimensions.
Process-Level Reward Models PRMs PRMBench step correctness fine-grained error detection simplicity soundness sensitivity critic models