MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process
  Errors Identification
Reasoning is an essential capacity for large language models (LLMs) to
address complex tasks, where the identification of process errors is vital for
improving this ability. Recently, process-level reward models (PRMs) were
proposed to provide step-wise rewards that facilitate reinforcement learning
and data production during training and guide LLMs toward correct steps during
inference, thereby improving reasoning accuracy. However, existing benchmarks
of PRMs are text-based and focus on error detection, neglecting other scenarios
like reasoning search. To address this gap, we introduce MPBench, a
comprehensive, multi-task, multimodal benchmark designed to systematically
assess the effectiveness of PRMs in diverse scenarios. MPBench employs three
evaluation paradigms, each targeting a specific role of PRMs in the reasoning
process: (1) Step Correctness, which assesses the correctness of each
intermediate reasoning step; (2) Answer Aggregation, which aggregates multiple
solutions and selects the best one; and (3) Reasoning Process Search, which
guides the search for optimal reasoning steps during inference. Through these
paradigms, MPBench makes comprehensive evaluations and provides insights into
the development of multimodal PRMs.
67d9442ec37d05ff0ab28e44
Zhaopan Xu
admin_assigned
2025-03-19T13:36:05.632Z
646dee28174cc96d50951991
/avatars/17d88a24c905e9819268b27037015a35.svg
xu
user
xuzhaopan
67d9442ec37d05ff0ab28e45
Pengfei Zhou
67d9442ec37d05ff0ab28e46
Jiaxin Ai
67d9442ec37d05ff0ab28e47
Wangbo Zhao
admin_assigned
2025-03-19T13:36:30.319Z
65d8b116f5d21ab1939efe03
/avatars/54d84f74faa2cc6fb8d6b58573dfde6d.svg
Wangbo Zhao
user
heisejiasuo
67d9442ec37d05ff0ab28e48
Kai Wang
67d9442ec37d05ff0ab28e49
Xiaojiang Peng
admin_assigned
2025-03-19T13:36:37.040Z
64acc5b3f5d40283c620d625
/avatars/1e7f488db184f445f7d93a37f6470590.svg
Xiaojiang Peng
user
xpeng-sztu
67d9442ec37d05ff0ab28e4a
Wenqi Shao
admin_assigned
2025-03-19T13:36:43.155Z
64b3fd42eec33e27dcc4c941
/avatars/5aa1a99468fa61d4b8b0e80b592c4e55.svg
Wenqi Shao
user
wqshao126
67d9442ec37d05ff0ab28e4b
Hongxun Yao
67d9442ec37d05ff0ab28e4c
Kaipeng Zhang
admin_assigned
2025-03-19T13:37:10.793Z
65f1713552c38a91e0a445e8
/avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
kaipeng
user
kpzhang996
MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process
  Errors Identification
Reasoning is an essential capacity for large language models (LLMs) to
address complex tasks, where the identification of process errors is vital for
improving this ability. Recently, process-level reward models (PRMs) were
proposed to provide step-wise rewards that facilitate reinforcement learning
and data production during training and guide LLMs toward correct steps during
inference, thereby improving reasoning accuracy. However, existing benchmarks
of PRMs are text-based and focus on error detection, neglecting other scenarios
like reasoning search. To address this gap, we introduce MPBench, a
comprehensive, multi-task, multimodal benchmark designed to systematically
assess the effectiveness of PRMs in diverse scenarios. MPBench employs three
evaluation paradigms, each targeting a specific role of PRMs in the reasoning
process: (1) Step Correctness, which assesses the correctness of each
intermediate reasoning step; (2) Answer Aggregation, which aggregates multiple
solutions and selects the best one; and (3) Reasoning Process Search, which
guides the search for optimal reasoning steps during inference. Through these
paradigms, MPBench makes comprehensive evaluations and provides insights into
the development of multimodal PRMs.
MPBench evaluates process-level reward models across step correctness, answer aggregation, and reasoning process search, addressing a gap in existing benchmarks.
large language models (LLMs) process-level reward models (PRMs) reinforcement learning data production reasoning accuracy MPBench multi-task multimodal step correctness answer aggregation reasoning process search