title: Dynamic Scaling of Unit Tests for Code Reward Modeling
summary: Current large language models (LLMs) often struggle to produce accurate
  responses on the first attempt for complex reasoning tasks like code
  generation. Prior research tackles this challenge by generating multiple
  candidate solutions and validating them with LLM-generated unit tests. The
  execution results of unit tests serve as reward signals to identify correct
  solutions. As LLMs always confidently make mistakes, these unit tests are not
  reliable, thereby diminishing the quality of reward signals. Motivated by the
  observation that scaling the number of solutions improves LLM performance, we
  explore the impact of scaling unit tests to enhance reward signal quality. Our
  pioneer experiment reveals a positive correlation between the number of unit
  tests and reward signal quality, with greater benefits observed in more
  challenging problems. Based on these insights, we propose CodeRM-8B, a
  lightweight yet effective unit test generator that enables efficient and
  high-quality unit test scaling. Additionally, we implement a dynamic scaling
  mechanism that adapts the number of unit tests based on problem difficulty,
  further improving efficiency. Experimental results show that our approach
  significantly improves performance across various models on three benchmarks
  (e.g., with gains of 18.43% for Llama3-8B and 3.42% for GPT-4o-mini on
  HumanEval Plus).
paper_authors._id: 67774afb5c6bccc41be7ba19
paper_authors.name: Zeyao Ma
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-03T02:27:07.916Z
paper_authors._id: 6384c07fdfffab4824ff45fb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669644372381-noauth.jpeg
paper_authors.fullname: Zeyao Ma
paper_authors.type: user
paper_authors.user: KAKA22
paper_authors._id: 67774afb5c6bccc41be7ba1a
paper_authors.name: Xiaokang Zhang
paper_authors._id: 67774afb5c6bccc41be7ba1b
paper_authors.name: Jing Zhang
paper_authors._id: 67774afb5c6bccc41be7ba1c
paper_authors.name: Jifan Yu
paper_authors._id: 67774afb5c6bccc41be7ba1d
paper_authors.name: Sijia Luo
paper_authors._id: 67774afb5c6bccc41be7ba1e
paper_authors.name: Jie Tang
paper_title: Dynamic Scaling of Unit Tests for Code Reward Modeling
paper_summary: Current large language models (LLMs) often struggle to produce accurate
  responses on the first attempt for complex reasoning tasks like code
  generation. Prior research tackles this challenge by generating multiple
  candidate solutions and validating them with LLM-generated unit tests. The
  execution results of unit tests serve as reward signals to identify correct
  solutions. As LLMs always confidently make mistakes, these unit tests are not
  reliable, thereby diminishing the quality of reward signals. Motivated by the
  observation that scaling the number of solutions improves LLM performance, we
  explore the impact of scaling unit tests to enhance reward signal quality. Our
  pioneer experiment reveals a positive correlation between the number of unit
  tests and reward signal quality, with greater benefits observed in more
  challenging problems. Based on these insights, we propose CodeRM-8B, a
  lightweight yet effective unit test generator that enables efficient and
  high-quality unit test scaling. Additionally, we implement a dynamic scaling
  mechanism that adapts the number of unit tests based on problem difficulty,
  further improving efficiency. Experimental results show that our approach
  significantly improves performance across various models on three benchmarks
  (e.g., with gains of 18.43% for Llama3-8B and 3.42% for GPT-4o-mini on
  HumanEval Plus).
paper_ai_keywords: large language models (LLMs) code generation multiple candidate solutions LLM-generated unit tests reward signals dynamic scaling mechanism problem difficulty CodeRM-8B lightweight yet effective unit test generator HumanEval Plus