title: ProcessBench: Identifying Process Errors in Mathematical Reasoning
summary: As language models regularly make mistakes when solving math problems,
  automated identification of errors in the reasoning process becomes
  increasingly significant for their scalable oversight. In this paper, we
  introduce ProcessBench for measuring the ability to identify erroneous steps in
  mathematical reasoning. It consists of 3,400 test cases, primarily focused on
  competition- and Olympiad-level math problems. Each test case contains a
  step-by-step solution with error location annotated by human experts. Models
  are required to identify the earliest step that contains an error, or conclude
  that all steps are correct. We conduct extensive evaluation on ProcessBench,
  involving two types of models: process reward models (PRMs) and critic models,
  where for the latter we prompt general language models to critique each
  solution step by step. We draw two main observations: (1) Existing PRMs
  typically fail to generalize to more challenging math problems beyond GSM8K and
  MATH. They underperform both critic models (i.e., prompted general language
  models) and our own trained PRM that is straightforwardly fine-tuned on the
  PRM800K dataset. (2) The best open-source model, QwQ-32B-Preview, has
  demonstrated the critique capability competitive with the proprietary model
  GPT-4o, despite that it still lags behind the reasoning-specialized o1-mini. We
  hope ProcessBench can foster future research in reasoning process assessment,
  paving the way toward scalable oversight of language models.
paper_authors._id: 6757e135887c48188a1f5015
paper_authors.name: Chujie Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-10T10:25:17.928Z
paper_authors._id: 610b70452719facd4ea85e28
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/610b70452719facd4ea85e28/S7nMy7D0Rxq0VIVblhYDG.jpeg
paper_authors.fullname: Chujie Zheng
paper_authors.type: user
paper_authors.user: chujiezheng
paper_authors._id: 6757e135887c48188a1f5016
paper_authors.name: Zhenru Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T16:13:54.636Z
paper_authors._id: 652f13fa6504631c10f308ed
paper_authors.avatarUrl: /avatars/440de01158c1d9152afbf33617fe27d1.svg
paper_authors.fullname: zhenrui
paper_authors.type: user
paper_authors.user: zhenrui
paper_authors._id: 6757e135887c48188a1f5017
paper_authors.name: Beichen Zhang
paper_authors._id: 6757e135887c48188a1f5018
paper_authors.name: Runji Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-10T10:25:16.002Z
paper_authors._id: 649a52e5de0fb7f3f499e583
paper_authors.avatarUrl: /avatars/25f6106fa168ae57ad5cd8ef55c70d31.svg
paper_authors.fullname: Runji Lin
paper_authors.type: user
paper_authors.user: RunjiLin
paper_authors._id: 6757e135887c48188a1f5019
paper_authors.name: Keming Lu
paper_authors._id: 6757e135887c48188a1f501a
paper_authors.name: Bowen Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T16:12:57.169Z
paper_authors._id: 6438b43ab2ea24b52ebac2b9
paper_authors.avatarUrl: /avatars/84133cd719a4b1e2f5c1a74178425f86.svg
paper_authors.fullname: Bowen Yu
paper_authors.type: user
paper_authors.user: bwy
paper_authors._id: 6757e135887c48188a1f501b
paper_authors.name: Dayiheng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T16:12:41.742Z
paper_authors._id: 6434d4989bd5a84b5dd0b0f5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6434d4989bd5a84b5dd0b0f5/0Elf9qbfG9Hkgypm9pTGm.jpeg
paper_authors.fullname: Dayiheng Liu
paper_authors.type: user
paper_authors.user: Losin94
paper_authors._id: 6757e135887c48188a1f501c
paper_authors.name: Jingren Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T16:12:36.306Z
paper_authors._id: 602f88f5e8149a962412a667
paper_authors.avatarUrl: /avatars/b78f0e583df8e5d5e3365934fe5f4900.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Jingren
paper_authors._id: 6757e135887c48188a1f501d
paper_authors.name: Junyang Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T16:12:26.676Z
paper_authors._id: 620760a26e3b7210c2ff1943
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
paper_authors.fullname: Junyang Lin
paper_authors.type: user
paper_authors.user: JustinLin610
paper_title: ProcessBench: Identifying Process Errors in Mathematical Reasoning
paper_summary: As language models regularly make mistakes when solving math problems,
  automated identification of errors in the reasoning process becomes
  increasingly significant for their scalable oversight. In this paper, we
  introduce ProcessBench for measuring the ability to identify erroneous steps in
  mathematical reasoning. It consists of 3,400 test cases, primarily focused on
  competition- and Olympiad-level math problems. Each test case contains a
  step-by-step solution with error location annotated by human experts. Models
  are required to identify the earliest step that contains an error, or conclude
  that all steps are correct. We conduct extensive evaluation on ProcessBench,
  involving two types of models: process reward models (PRMs) and critic models,
  where for the latter we prompt general language models to critique each
  solution step by step. We draw two main observations: (1) Existing PRMs
  typically fail to generalize to more challenging math problems beyond GSM8K and
  MATH. They underperform both critic models (i.e., prompted general language
  models) and our own trained PRM that is straightforwardly fine-tuned on the
  PRM800K dataset. (2) The best open-source model, QwQ-32B-Preview, has
  demonstrated the critique capability competitive with the proprietary model
  GPT-4o, despite that it still lags behind the reasoning-specialized o1-mini. We
  hope ProcessBench can foster future research in reasoning process assessment,
  paving the way toward scalable oversight of language models.
paper_ai_summary: ProcessBench evaluates models' ability to identify errors in mathematical reasoning steps, showing that existing process reward models struggle with difficult problems and underperform compared to critic models and a fine-tuned PRM.
paper_ai_keywords: ProcessBench process reward models critic models error identification mathematical reasoning PRM800K dataset QwQ-32B-Preview GPT-4o reasoning-specialized o1-mini