title: MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable
    Step-Level Supervision
summary: While Multimodal Large Language Models (MLLMs) have achieved impressive
  progress in vision-language understanding, they still struggle with complex
  multi-step reasoning, often producing logically inconsistent or partially
  correct solutions. A key limitation lies in the lack of fine-grained
  supervision over intermediate reasoning steps. To address this, we propose
  MM-PRM, a process reward model trained within a fully automated, scalable
  framework. We first build MM-Policy, a strong multimodal model trained on
  diverse mathematical reasoning data. Then, we construct MM-K12, a curated
  dataset of 10,000 multimodal math problems with verifiable answers, which
  serves as seed data. Leveraging a Monte Carlo Tree Search (MCTS)-based
  pipeline, we generate over 700k step-level annotations without human labeling.
  The resulting PRM is used to score candidate reasoning paths in the Best-of-N
  inference setup and achieves significant improvements across both in-domain
  (MM-K12 test set) and out-of-domain (OlympiadBench, MathVista, etc.)
  benchmarks. Further analysis confirms the effectiveness of soft labels, smaller
  learning rates, and path diversity in optimizing PRM performance. MM-PRM
  demonstrates that process supervision is a powerful tool for enhancing the
  logical robustness of multimodal reasoning systems. We release all our codes
  and data at https://github.com/ModalMinds/MM-PRM.
paper_authors._id: 682bfa77444a7d5f589a8769
paper_authors.name: Lingxiao Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:27:27.006Z
paper_authors._id: 666fe1a5b07525f0bde69c27
paper_authors.avatarUrl: /avatars/bb98ab0b974c8fe011739baa8dadd91a.svg
paper_authors.fullname: Lingxiao Du
paper_authors.type: user
paper_authors.user: Cierra0506
paper_authors._id: 682bfa77444a7d5f589a876a
paper_authors.name: Fanqing Meng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:27:33.802Z
paper_authors._id: 640b37b2bab5ca8fbe7df8f2
paper_authors.avatarUrl: /avatars/c7bef45efad6a0d911a720e2236fcba5.svg
paper_authors.fullname: fanqing meng
paper_authors.type: user
paper_authors.user: FanqingM
paper_authors._id: 682bfa77444a7d5f589a876b
paper_authors.name: Zongkai Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T09:03:38.181Z
paper_authors._id: 674d42a03a4b7e31a1707218
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/3DIez-RYnDMYe1U-m0qBZ.png
paper_authors.fullname: kkkai
paper_authors.type: user
paper_authors.user: Zkkkai
paper_authors._id: 682bfa77444a7d5f589a876c
paper_authors.name: Zhixiang Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:28:03.869Z
paper_authors._id: 674bfdf227f531cdc248bb5c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/674bfdf227f531cdc248bb5c/xh4gw89sr8MzNzRdiTjFx.jpeg
paper_authors.fullname: Zhixiang Zhou
paper_authors.type: user
paper_authors.user: SuperposedWave
paper_authors._id: 682bfa77444a7d5f589a876d
paper_authors.name: Ping Luo
paper_authors._id: 682bfa77444a7d5f589a876e
paper_authors.name: Qiaosheng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:28:11.669Z
paper_authors._id: 63cf4ecdc1dedf59c8f8362e
paper_authors.avatarUrl: /avatars/cede885854d6a1551860080d55c87568.svg
paper_authors.fullname: Qiaosheng ZHANG
paper_authors.type: user
paper_authors.user: Domingo12
paper_authors._id: 682bfa77444a7d5f589a876f
paper_authors.name: Wenqi Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:28:17.437Z
paper_authors._id: 64b3fd42eec33e27dcc4c941
paper_authors.avatarUrl: /avatars/5aa1a99468fa61d4b8b0e80b592c4e55.svg
paper_authors.fullname: Wenqi Shao
paper_authors.type: user
paper_authors.user: wqshao126
paper_title: MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable
    Step-Level Supervision
paper_summary: While Multimodal Large Language Models (MLLMs) have achieved impressive
  progress in vision-language understanding, they still struggle with complex
  multi-step reasoning, often producing logically inconsistent or partially
  correct solutions. A key limitation lies in the lack of fine-grained
  supervision over intermediate reasoning steps. To address this, we propose
  MM-PRM, a process reward model trained within a fully automated, scalable
  framework. We first build MM-Policy, a strong multimodal model trained on
  diverse mathematical reasoning data. Then, we construct MM-K12, a curated
  dataset of 10,000 multimodal math problems with verifiable answers, which
  serves as seed data. Leveraging a Monte Carlo Tree Search (MCTS)-based
  pipeline, we generate over 700k step-level annotations without human labeling.
  The resulting PRM is used to score candidate reasoning paths in the Best-of-N
  inference setup and achieves significant improvements across both in-domain
  (MM-K12 test set) and out-of-domain (OlympiadBench, MathVista, etc.)
  benchmarks. Further analysis confirms the effectiveness of soft labels, smaller
  learning rates, and path diversity in optimizing PRM performance. MM-PRM
  demonstrates that process supervision is a powerful tool for enhancing the
  logical robustness of multimodal reasoning systems. We release all our codes
  and data at https://github.com/ModalMinds/MM-PRM.
paper_ai_summary: MM-PRM, a process reward model with step-level annotations, enhances logical reasoning in multimodal language models by using automated supervision and achieves improved performance on various benchmarks.
paper_ai_keywords: Multimodal Large Language Models (MLLMs) MM-Policy MM-K12 Monte Carlo Tree Search (MCTS) process reward model Best-of-N inference logical robustness multimodal reasoning systems soft labels path diversity