title: Outcome-Refining Process Supervision for Code Generation
summary: Large Language Models have demonstrated remarkable capabilities in code
  generation, yet they often struggle with complex programming tasks that require
  deep algorithmic reasoning. While process supervision through learned reward
  models shows promise in guiding reasoning steps, it requires expensive training
  data and suffers from unreliable evaluation. We propose Outcome-Refining
  Process Supervision, a novel paradigm that treats outcome refinement itself as
  the process to be supervised. Our framework leverages concrete execution
  signals to ground the supervision of reasoning steps, while using
  tree-structured exploration to maintain multiple solution trajectories
  simultaneously. Experiments demonstrate that our approach enables even smaller
  models to achieve high success accuracy and performance metrics on competitive
  programming tasks, creates more reliable verification than traditional reward
  models without requiring training PRMs. Our approach achieves significant
  improvements across 5 models and 3 datasets: an average of 26.9% increase in
  correctness and 42.2% in efficiency. The results suggest that providing
  structured reasoning space with concrete verification signals is crucial for
  solving complex programming tasks. We open-source all our code and data at:
  https://github.com/zhuohaoyu/ORPS
paper_authors._id: 676a444d9315022860ac1f0e
paper_authors.name: Zhuohao Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:34:53.843Z
paper_authors._id: 61e24808b31e7cc38eb84d37
paper_authors.avatarUrl: /avatars/65fbea940fad211462ecc5ad725e0c28.svg
paper_authors.fullname: Zhuohao Yu
paper_authors.type: user
paper_authors.user: zhuohaoyu
paper_authors._id: 676a444d9315022860ac1f0f
paper_authors.name: Weizheng Gu
paper_authors._id: 676a444d9315022860ac1f10
paper_authors.name: Yidong Wang
paper_authors._id: 676a444d9315022860ac1f11
paper_authors.name: Zhengran Zeng
paper_authors._id: 676a444d9315022860ac1f12
paper_authors.name: Jindong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:54:33.006Z
paper_authors._id: 6204cc0d522e40b4a18d86e2
paper_authors.avatarUrl: /avatars/18daf2de5671e711dc745388dd60569d.svg
paper_authors.fullname: Jindong Wang
paper_authors.type: user
paper_authors.user: jindongwang
paper_authors._id: 676a444d9315022860ac1f13
paper_authors.name: Wei Ye
paper_authors._id: 676a444d9315022860ac1f14
paper_authors.name: Shikun Zhang
paper_title: Outcome-Refining Process Supervision for Code Generation
paper_summary: Large Language Models have demonstrated remarkable capabilities in code
  generation, yet they often struggle with complex programming tasks that require
  deep algorithmic reasoning. While process supervision through learned reward
  models shows promise in guiding reasoning steps, it requires expensive training
  data and suffers from unreliable evaluation. We propose Outcome-Refining
  Process Supervision, a novel paradigm that treats outcome refinement itself as
  the process to be supervised. Our framework leverages concrete execution
  signals to ground the supervision of reasoning steps, while using
  tree-structured exploration to maintain multiple solution trajectories
  simultaneously. Experiments demonstrate that our approach enables even smaller
  models to achieve high success accuracy and performance metrics on competitive
  programming tasks, creates more reliable verification than traditional reward
  models without requiring training PRMs. Our approach achieves significant
  improvements across 5 models and 3 datasets: an average of 26.9% increase in
  correctness and 42.2% in efficiency. The results suggest that providing
  structured reasoning space with concrete verification signals is crucial for
  solving complex programming tasks. We open-source all our code and data at:
  https://github.com/zhuohaoyu/ORPS
paper_ai_summary: Outcome-Refining Process Supervision uses concrete execution signals and tree-structured exploration to enhance algorithmic reasoning in code generation tasks.
paper_ai_keywords: outcome refinement process supervision learned reward models concrete execution signals tree-structured exploration algorithmic reasoning code generation competitive programming tasks verification