title: Scaling Relationship on Learning Mathematical Reasoning with Large
    Language Models
summary: Mathematical reasoning is a challenging task for large language models
  (LLMs), while the scaling relationship of it with respect to LLM capacity is
  under-explored. In this paper, we investigate how the pre-training loss,
  supervised data amount, and augmented data amount influence the reasoning
  performances of a supervised LLM. We find that pre-training loss is a better
  indicator of the model's performance than the model's parameter count. We apply
  supervised fine-tuning (SFT) with different amounts of supervised data and
  empirically find a log-linear relation between data amount and model
  performance, and we find better models improve less with enlarged supervised
  datasets. To augment more data samples for improving model performances without
  any human effort, we propose to apply Rejection sampling Fine-Tuning (RFT). RFT
  uses supervised models to generate and collect correct reasoning paths as
  augmented fine-tuning datasets. We find with augmented samples containing more
  distinct reasoning paths, RFT improves mathematical reasoning performance more
  for LLMs. We also find RFT brings more improvement for less performant LLMs.
  Furthermore, we combine rejection samples from multiple models which push
  LLaMA-7B to an accuracy of 49.3% and outperforms the supervised fine-tuning
  (SFT) accuracy of 35.9% significantly.
paper_authors._id: 64cc5655a257a3212c1588d4
paper_authors.name: Zheng Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-04T11:40:20.892Z
paper_authors._id: 5f694b8b5e78cc6b0ed31bd2
paper_authors.avatarUrl: /avatars/63f21d30918c34fff9ea592e1039b3f0.svg
paper_authors.fullname: Zheng Yuan
paper_authors.type: user
paper_authors.user: GanjinZero
paper_authors._id: 64cc5655a257a3212c1588d5
paper_authors.name: Hongyi Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-04T15:05:10.506Z
paper_authors._id: 64b77b602f5a966b9731b14a
paper_authors.avatarUrl: /avatars/cdcf534bd451140dff47286095336535.svg
paper_authors.fullname: Hongyi Yuan
paper_authors.type: user
paper_authors.user: Yuan1997
paper_authors._id: 64cc5655a257a3212c1588d6
paper_authors.name: Chengpeng Li
paper_authors._id: 64cc5655a257a3212c1588d7
paper_authors.name: Guanting Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-15T08:29:15.654Z
paper_authors._id: 61cd4b833dd34ba1985e0753
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61cd4b833dd34ba1985e0753/BfHfrwotoMESpXZOHiIe4.png
paper_authors.fullname: KABI
paper_authors.type: user
paper_authors.user: dongguanting
paper_authors._id: 64cc5655a257a3212c1588d8
paper_authors.name: Chuanqi Tan
paper_authors._id: 64cc5655a257a3212c1588d9
paper_authors.name: Chang Zhou
paper_title: Scaling Relationship on Learning Mathematical Reasoning with Large
    Language Models
paper_summary: Mathematical reasoning is a challenging task for large language models
  (LLMs), while the scaling relationship of it with respect to LLM capacity is
  under-explored. In this paper, we investigate how the pre-training loss,
  supervised data amount, and augmented data amount influence the reasoning
  performances of a supervised LLM. We find that pre-training loss is a better
  indicator of the model's performance than the model's parameter count. We apply
  supervised fine-tuning (SFT) with different amounts of supervised data and
  empirically find a log-linear relation between data amount and model
  performance, and we find better models improve less with enlarged supervised
  datasets. To augment more data samples for improving model performances without
  any human effort, we propose to apply Rejection sampling Fine-Tuning (RFT). RFT
  uses supervised models to generate and collect correct reasoning paths as
  augmented fine-tuning datasets. We find with augmented samples containing more
  distinct reasoning paths, RFT improves mathematical reasoning performance more
  for LLMs. We also find RFT brings more improvement for less performant LLMs.
  Furthermore, we combine rejection samples from multiple models which push
  LLaMA-7B to an accuracy of 49.3% and outperforms the supervised fine-tuning
  (SFT) accuracy of 35.9% significantly.
paper_ai_keywords: supervised fine-tuning (SFT) Rejection sampling Fine-Tuning (RFT) reasoning paths mathematical reasoning performance distinct reasoning paths