title: Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
summary: Reinforcement finetuning (RFT) has shown great potential for enhancing the
  mathematical reasoning capabilities of large language models (LLMs), but it is
  often sample- and compute-inefficient, requiring extensive training. In this
  work, we introduce AdaRFT (Adaptive Curriculum Reinforcement Finetuning), a
  method that significantly improves both the efficiency and final accuracy of
  RFT through adaptive curriculum learning. AdaRFT dynamically adjusts the
  difficulty of training problems based on the model's recent reward signals,
  ensuring that the model consistently trains on tasks that are challenging but
  solvable. This adaptive sampling strategy accelerates learning by maintaining
  an optimal difficulty range, avoiding wasted computation on problems that are
  too easy or too hard. AdaRFT requires only a lightweight extension to standard
  RFT algorithms like Proximal Policy Optimization (PPO), without modifying the
  reward function or model architecture. Experiments on competition-level math
  datasets-including AMC, AIME, and IMO-style problems-demonstrate that AdaRFT
  significantly improves both training efficiency and reasoning performance. We
  evaluate AdaRFT across multiple data distributions and model sizes, showing
  that it reduces the number of training steps by up to 2x and improves accuracy
  by a considerable margin, offering a more scalable and effective RFT framework.
paper_authors._id: 67f5e81488885c9950c7ab7e
paper_authors.name: Taiwei Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-09T07:34:46.737Z
paper_authors._id: 62e1b3cb3eb0730f621a83f6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1658958764563-noauth.jpeg
paper_authors.fullname: Taiwei Shi
paper_authors.type: user
paper_authors.user: MaksimSTW
paper_authors._id: 67f5e81488885c9950c7ab7f
paper_authors.name: Yiyang Wu
paper_authors._id: 67f5e81488885c9950c7ab80
paper_authors.name: Linxin Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T09:47:18.915Z
paper_authors._id: 64d660308ebc40443813f014
paper_authors.avatarUrl: /avatars/516bb2d2383be99794e366dfb41636b6.svg
paper_authors.fullname: Linxin Song
paper_authors.type: user
paper_authors.user: linxinso
paper_authors._id: 67f5e81488885c9950c7ab81
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T09:24:20.414Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_authors._id: 67f5e81488885c9950c7ab82
paper_authors.name: Jieyu Zhao
paper_title: Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
paper_summary: Reinforcement finetuning (RFT) has shown great potential for enhancing the
  mathematical reasoning capabilities of large language models (LLMs), but it is
  often sample- and compute-inefficient, requiring extensive training. In this
  work, we introduce AdaRFT (Adaptive Curriculum Reinforcement Finetuning), a
  method that significantly improves both the efficiency and final accuracy of
  RFT through adaptive curriculum learning. AdaRFT dynamically adjusts the
  difficulty of training problems based on the model's recent reward signals,
  ensuring that the model consistently trains on tasks that are challenging but
  solvable. This adaptive sampling strategy accelerates learning by maintaining
  an optimal difficulty range, avoiding wasted computation on problems that are
  too easy or too hard. AdaRFT requires only a lightweight extension to standard
  RFT algorithms like Proximal Policy Optimization (PPO), without modifying the
  reward function or model architecture. Experiments on competition-level math
  datasets-including AMC, AIME, and IMO-style problems-demonstrate that AdaRFT
  significantly improves both training efficiency and reasoning performance. We
  evaluate AdaRFT across multiple data distributions and model sizes, showing
  that it reduces the number of training steps by up to 2x and improves accuracy
  by a considerable margin, offering a more scalable and effective RFT framework.
paper_ai_summary: AdaRFT uses adaptive curriculum learning to improve the efficiency and accuracy of reinforcement fine-tuning for large language models in mathematical reasoning.
paper_ai_keywords: Reinforcement finetuning (RFT) AdaRFT adaptive curriculum learning reward signals Proximal Policy Optimization (PPO) AMC AIME IMO training efficiency reasoning performance