title: MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process
    Errors Identification
summary: Reasoning is an essential capacity for large language models (LLMs) to
  address complex tasks, where the identification of process errors is vital for
  improving this ability. Recently, process-level reward models (PRMs) were
  proposed to provide step-wise rewards that facilitate reinforcement learning
  and data production during training and guide LLMs toward correct steps during
  inference, thereby improving reasoning accuracy. However, existing benchmarks
  of PRMs are text-based and focus on error detection, neglecting other scenarios
  like reasoning search. To address this gap, we introduce MPBench, a
  comprehensive, multi-task, multimodal benchmark designed to systematically
  assess the effectiveness of PRMs in diverse scenarios. MPBench employs three
  evaluation paradigms, each targeting a specific role of PRMs in the reasoning
  process: (1) Step Correctness, which assesses the correctness of each
  intermediate reasoning step; (2) Answer Aggregation, which aggregates multiple
  solutions and selects the best one; and (3) Reasoning Process Search, which
  guides the search for optimal reasoning steps during inference. Through these
  paradigms, MPBench makes comprehensive evaluations and provides insights into
  the development of multimodal PRMs.
paper_authors._id: 67d9442ec37d05ff0ab28e44
paper_authors.name: Zhaopan Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:36:05.632Z
paper_authors._id: 646dee28174cc96d50951991
paper_authors.avatarUrl: /avatars/17d88a24c905e9819268b27037015a35.svg
paper_authors.fullname: xu
paper_authors.type: user
paper_authors.user: xuzhaopan
paper_authors._id: 67d9442ec37d05ff0ab28e45
paper_authors.name: Pengfei Zhou
paper_authors._id: 67d9442ec37d05ff0ab28e46
paper_authors.name: Jiaxin Ai
paper_authors._id: 67d9442ec37d05ff0ab28e47
paper_authors.name: Wangbo Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:36:30.319Z
paper_authors._id: 65d8b116f5d21ab1939efe03
paper_authors.avatarUrl: /avatars/54d84f74faa2cc6fb8d6b58573dfde6d.svg
paper_authors.fullname: Wangbo Zhao
paper_authors.type: user
paper_authors.user: heisejiasuo
paper_authors._id: 67d9442ec37d05ff0ab28e48
paper_authors.name: Kai Wang
paper_authors._id: 67d9442ec37d05ff0ab28e49
paper_authors.name: Xiaojiang Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:36:37.040Z
paper_authors._id: 64acc5b3f5d40283c620d625
paper_authors.avatarUrl: /avatars/1e7f488db184f445f7d93a37f6470590.svg
paper_authors.fullname: Xiaojiang Peng
paper_authors.type: user
paper_authors.user: xpeng-sztu
paper_authors._id: 67d9442ec37d05ff0ab28e4a
paper_authors.name: Wenqi Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:36:43.155Z
paper_authors._id: 64b3fd42eec33e27dcc4c941
paper_authors.avatarUrl: /avatars/5aa1a99468fa61d4b8b0e80b592c4e55.svg
paper_authors.fullname: Wenqi Shao
paper_authors.type: user
paper_authors.user: wqshao126
paper_authors._id: 67d9442ec37d05ff0ab28e4b
paper_authors.name: Hongxun Yao
paper_authors._id: 67d9442ec37d05ff0ab28e4c
paper_authors.name: Kaipeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:37:10.793Z
paper_authors._id: 65f1713552c38a91e0a445e8
paper_authors.avatarUrl: /avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
paper_authors.fullname: kaipeng
paper_authors.type: user
paper_authors.user: kpzhang996
paper_title: MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process
    Errors Identification
paper_summary: Reasoning is an essential capacity for large language models (LLMs) to
  address complex tasks, where the identification of process errors is vital for
  improving this ability. Recently, process-level reward models (PRMs) were
  proposed to provide step-wise rewards that facilitate reinforcement learning
  and data production during training and guide LLMs toward correct steps during
  inference, thereby improving reasoning accuracy. However, existing benchmarks
  of PRMs are text-based and focus on error detection, neglecting other scenarios
  like reasoning search. To address this gap, we introduce MPBench, a
  comprehensive, multi-task, multimodal benchmark designed to systematically
  assess the effectiveness of PRMs in diverse scenarios. MPBench employs three
  evaluation paradigms, each targeting a specific role of PRMs in the reasoning
  process: (1) Step Correctness, which assesses the correctness of each
  intermediate reasoning step; (2) Answer Aggregation, which aggregates multiple
  solutions and selects the best one; and (3) Reasoning Process Search, which
  guides the search for optimal reasoning steps during inference. Through these
  paradigms, MPBench makes comprehensive evaluations and provides insights into
  the development of multimodal PRMs.
paper_ai_summary: MPBench evaluates process-level reward models across step correctness, answer aggregation, and reasoning process search, addressing a gap in existing benchmarks.
paper_ai_keywords: large language models (LLMs) process-level reward models (PRMs) reinforcement learning data production reasoning accuracy MPBench multi-task multimodal step correctness answer aggregation reasoning process search