title: Error Typing for Smarter Rewards: Improving Process Reward Models with
    Error-Aware Hierarchical Supervision
summary: Large Language Models (LLMs) are prone to hallucination, especially during
  multi-hop and reasoning-intensive tasks such as mathematical problem solving.
  While Outcome Reward Models verify only final answers, Process Reward Models
  (PRMs) score each intermediate step to steer generation toward coherent
  solutions. We introduce PathFinder-PRM, a novel hierarchical, error-aware
  discriminative PRM that first classifies math and consistency errors at each
  step, then combines these fine-grained signals to estimate step correctness. To
  train PathFinder-PRM, we construct a 400K-sample dataset by enriching the
  human-annotated PRM800K corpus and RLHFlow Mistral traces with
  three-dimensional step-level labels. On PRMBench, PathFinder-PRM achieves a new
  state-of-the-art PRMScore of 67.7, outperforming the prior best (65.5) while
  using 3 times less data. When applied to reward guided greedy search, our model
  yields prm@8 48.3, a +1.5 point gain over the strongest baseline. These results
  demonstrate that decoupled error detection and reward estimation not only boost
  fine-grained error detection but also substantially improve end-to-end,
  reward-guided mathematical reasoning with greater data efficiency.
paper_authors._id: 6835182873a16b09c94ac4d2
paper_authors.name: Tej Deep Pala
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T10:13:59.973Z
paper_authors._id: 633f875450d83f5065d459d6
paper_authors.avatarUrl: /avatars/48312867ac2c05d589869972f90881b9.svg
paper_authors.fullname: Pala Tej Deep
paper_authors.type: user
paper_authors.user: Tej3
paper_authors._id: 6835182873a16b09c94ac4d3
paper_authors.name: Panshul Sharma
paper_authors._id: 6835182873a16b09c94ac4d4
paper_authors.name: Amir Zadeh
paper_authors._id: 6835182873a16b09c94ac4d5
paper_authors.name: Chuan Li
paper_authors._id: 6835182873a16b09c94ac4d6
paper_authors.name: Soujanya Poria
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:46:25.138Z
paper_authors._id: 626b626405fe1cb65725aca1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/aa-Lata46I3fXOmMetvXH.jpeg
paper_authors.fullname: Soujanya Poria
paper_authors.type: user
paper_authors.user: soujanyaporia
paper_title: Error Typing for Smarter Rewards: Improving Process Reward Models with
    Error-Aware Hierarchical Supervision
paper_summary: Large Language Models (LLMs) are prone to hallucination, especially during
  multi-hop and reasoning-intensive tasks such as mathematical problem solving.
  While Outcome Reward Models verify only final answers, Process Reward Models
  (PRMs) score each intermediate step to steer generation toward coherent
  solutions. We introduce PathFinder-PRM, a novel hierarchical, error-aware
  discriminative PRM that first classifies math and consistency errors at each
  step, then combines these fine-grained signals to estimate step correctness. To
  train PathFinder-PRM, we construct a 400K-sample dataset by enriching the
  human-annotated PRM800K corpus and RLHFlow Mistral traces with
  three-dimensional step-level labels. On PRMBench, PathFinder-PRM achieves a new
  state-of-the-art PRMScore of 67.7, outperforming the prior best (65.5) while
  using 3 times less data. When applied to reward guided greedy search, our model
  yields prm@8 48.3, a +1.5 point gain over the strongest baseline. These results
  demonstrate that decoupled error detection and reward estimation not only boost
  fine-grained error detection but also substantially improve end-to-end,
  reward-guided mathematical reasoning with greater data efficiency.
paper_ai_summary: PathFinder-PRM, a hierarchical and error-aware Process Reward Model, improves mathematical problem-solving by fine-grained error classification and step correctness estimation, achieving state-of-the-art PRMScore with reduced data usage.
paper_ai_keywords: Large Language Models hallucination mathematical problem solving Outcome Reward Models Process Reward Models PathFinder-PRM hierarchical error-aware discriminative PRM math errors consistency errors step correctness PRMBench PRMScore reward guided greedy search prm@8 data efficiency