title: AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward
    Modeling
summary: In this paper, we introduce AceMath, a suite of frontier math models that
  excel in solving complex math problems, along with highly effective reward
  models capable of evaluating generated solutions and reliably identifying the
  correct ones. To develop the instruction-tuned math models, we propose a
  supervised fine-tuning (SFT) process that first achieves competitive
  performance across general domains, followed by targeted fine-tuning for the
  math domain using a carefully curated set of prompts and synthetically
  generated responses. The resulting model, AceMath-72B-Instruct greatly
  outperforms Qwen2.5-Math-72B-Instruct, GPT-4o and Claude-3.5 Sonnet. To develop
  math-specialized reward model, we first construct AceMath-RewardBench, a
  comprehensive and robust benchmark for evaluating math reward models across
  diverse problems and difficulty levels. After that, we present a systematic
  approach to build our math reward models. The resulting model, AceMath-72B-RM,
  consistently outperforms state-of-the-art reward models. Furthermore, when
  combining AceMath-72B-Instruct with AceMath-72B-RM, we achieve the highest
  average rm@8 score across the math reasoning benchmarks. We will release model
  weights, training data, and evaluation benchmarks at:
  https://research.nvidia.com/labs/adlr/acemath
paper_authors._id: 6764e3e30afbb34519fd2018
paper_authors.name: Zihan Liu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-20T03:26:28.621Z
paper_authors._id: 65f33b1c9f7970ccc0234cbf
paper_authors.avatarUrl: /avatars/99fbab303912e3674663251c04279907.svg
paper_authors.fullname: Zihan Liu
paper_authors.type: user
paper_authors.user: zihanliu
paper_authors._id: 6764e3e30afbb34519fd2019
paper_authors.name: Yang Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:32:40.034Z
paper_authors._id: 62bc9d90e81dfd65cced9316
paper_authors.avatarUrl: /avatars/05df14cd1fdbc7d6a80d2960a05a94f0.svg
paper_authors.fullname: Yang Chen
paper_authors.type: user
paper_authors.user: ychenNLP
paper_authors._id: 6764e3e30afbb34519fd201a
paper_authors.name: Mohammad Shoeybi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:27:29.660Z
paper_authors._id: 6641544c695975af2cbd0da6
paper_authors.avatarUrl: /avatars/0ad3c18dcba585259b064fe9b00a07ce.svg
paper_authors.fullname: Mohammad Shoeybi
paper_authors.type: user
paper_authors.user: shoeybi
paper_authors._id: 6764e3e30afbb34519fd201b
paper_authors.name: Bryan Catanzaro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:27:23.912Z
paper_authors._id: 6311021788942700629e6247
paper_authors.avatarUrl: /avatars/e7adc1632b76e80e7e4a590033d1c20a.svg
paper_authors.fullname: Bryan Catanzaro
paper_authors.type: user
paper_authors.user: ctnzr
paper_authors._id: 6764e3e30afbb34519fd201c
paper_authors.name: Wei Ping
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-20T03:26:28.621Z
paper_authors._id: 663ee43bfeeb49803537da98
paper_authors.avatarUrl: /avatars/17c3e9c435cc36fb04b4589e6176a243.svg
paper_authors.fullname: Wei Ping
paper_authors.type: user
paper_authors.user: wping
paper_title: AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward
    Modeling
paper_summary: In this paper, we introduce AceMath, a suite of frontier math models that
  excel in solving complex math problems, along with highly effective reward
  models capable of evaluating generated solutions and reliably identifying the
  correct ones. To develop the instruction-tuned math models, we propose a
  supervised fine-tuning (SFT) process that first achieves competitive
  performance across general domains, followed by targeted fine-tuning for the
  math domain using a carefully curated set of prompts and synthetically
  generated responses. The resulting model, AceMath-72B-Instruct greatly
  outperforms Qwen2.5-Math-72B-Instruct, GPT-4o and Claude-3.5 Sonnet. To develop
  math-specialized reward model, we first construct AceMath-RewardBench, a
  comprehensive and robust benchmark for evaluating math reward models across
  diverse problems and difficulty levels. After that, we present a systematic
  approach to build our math reward models. The resulting model, AceMath-72B-RM,
  consistently outperforms state-of-the-art reward models. Furthermore, when
  combining AceMath-72B-Instruct with AceMath-72B-RM, we achieve the highest
  average rm@8 score across the math reasoning benchmarks. We will release model
  weights, training data, and evaluation benchmarks at:
  https://research.nvidia.com/labs/adlr/acemath
paper_ai_keywords: supervised fine-tuning (SFT) instruction-tuned math models fine-tuning prompts synthetically generated responses vision-and-language models reward models AceMath-RewardBench benchmark math reasoning benchmarks ACEMath-72B-Instruct Qwen2.5-Math-72B-Instruct GPT-4o Claude-3.5 mathematical problem span highlighting feature extraction feature representation rm@8 score model evaluation