title: Warm Up Before You Train: Unlocking General Reasoning in
    Resource-Constrained Settings
summary: Designing effective reasoning-capable LLMs typically requires training using
  Reinforcement Learning with Verifiable Rewards (RLVR) or distillation with
  carefully curated Long Chain of Thoughts (CoT), both of which depend heavily on
  extensive training data. This creates a major challenge when the amount of
  quality training data is scarce. We propose a sample-efficient, two-stage
  training strategy to develop reasoning LLMs under limited supervision. In the
  first stage, we "warm up" the model by distilling Long CoTs from a toy domain,
  namely, Knights \& Knaves (K\&K) logic puzzles to acquire general reasoning
  skills. In the second stage, we apply RLVR to the warmed-up model using a
  limited set of target-domain examples. Our experiments demonstrate that this
  two-phase approach offers several benefits: (i) the warmup phase alone
  facilitates generalized reasoning, leading to performance improvements across a
  range of tasks, including MATH, HumanEval^{+}, and MMLU-Pro. (ii) When both
  the base model and the warmed-up model are RLVR trained on the same small
  dataset (leq100 examples), the warmed-up model consistently outperforms the
  base model; (iii) Warming up before RLVR training allows a model to maintain
  cross-domain generalizability even after training on a specific domain; (iv)
  Introducing warmup in the pipeline improves not only accuracy but also overall
  sample efficiency during RLVR training. The results in this paper highlight the
  promise of warmup for building robust reasoning LLMs in data-scarce
  environments.
paper_authors._id: 682d6ccc26146f27d1ab3d83
paper_authors.name: Safal Shrestha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T08:40:07.374Z
paper_authors._id: 64cb922ec7f30fbf7b91a9a7
paper_authors.avatarUrl: /avatars/457eae5e56b9641ee5543146447d1755.svg
paper_authors.fullname: Safal Shrestha
paper_authors.type: user
paper_authors.user: safal312
paper_authors._id: 682d6ccc26146f27d1ab3d84
paper_authors.name: Minwu Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:27:16.736Z
paper_authors._id: 64e77b47d96966317b45eeb3
paper_authors.avatarUrl: /avatars/6b67eba3f15d6cd86ac3ad55c1daf166.svg
paper_authors.fullname: Minwu Kim
paper_authors.type: user
paper_authors.user: guactastesgood
paper_authors._id: 682d6ccc26146f27d1ab3d85
paper_authors.name: Aadim Nepal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:27:23.895Z
paper_authors._id: 67b5b0800918c8645fea09d7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/_6pKdkxjsC0dOBbKAN3ok.png
paper_authors.fullname: Aadim Nepal
paper_authors.type: user
paper_authors.user: AadimNepal
paper_authors._id: 682d6ccc26146f27d1ab3d86
paper_authors.name: Anubhav Shrestha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:27:30.010Z
paper_authors._id: 66a0394001cfae79a11b54ea
paper_authors.avatarUrl: /avatars/e135cca9b9027cfb762e7ff86bd5ab5a.svg
paper_authors.fullname: Anubhav Shrestha
paper_authors.type: user
paper_authors.user: xanubhav81
paper_authors._id: 682d6ccc26146f27d1ab3d87
paper_authors.name: Keith Ross
paper_title: Warm Up Before You Train: Unlocking General Reasoning in
    Resource-Constrained Settings
paper_summary: Designing effective reasoning-capable LLMs typically requires training using
  Reinforcement Learning with Verifiable Rewards (RLVR) or distillation with
  carefully curated Long Chain of Thoughts (CoT), both of which depend heavily on
  extensive training data. This creates a major challenge when the amount of
  quality training data is scarce. We propose a sample-efficient, two-stage
  training strategy to develop reasoning LLMs under limited supervision. In the
  first stage, we "warm up" the model by distilling Long CoTs from a toy domain,
  namely, Knights \& Knaves (K\&K) logic puzzles to acquire general reasoning
  skills. In the second stage, we apply RLVR to the warmed-up model using a
  limited set of target-domain examples. Our experiments demonstrate that this
  two-phase approach offers several benefits: (i) the warmup phase alone
  facilitates generalized reasoning, leading to performance improvements across a
  range of tasks, including MATH, HumanEval^{+}, and MMLU-Pro. (ii) When both
  the base model and the warmed-up model are RLVR trained on the same small
  dataset (leq100 examples), the warmed-up model consistently outperforms the
  base model; (iii) Warming up before RLVR training allows a model to maintain
  cross-domain generalizability even after training on a specific domain; (iv)
  Introducing warmup in the pipeline improves not only accuracy but also overall
  sample efficiency during RLVR training. The results in this paper highlight the
  promise of warmup for building robust reasoning LLMs in data-scarce
  environments.
paper_ai_summary: A two-stage training strategy involving a warm-up phase with distillation of logic puzzles followed by RLVR training on limited domain-specific data improves reasoning capabilities and sample efficiency in LLMs.
paper_ai_keywords: Reinforcement Learning with Verifiable Rewards (RLVR) Long Chain of Thoughts (CoT) Knights & Knaves (K&K) logic puzzles warm-up phase generalized reasoning MATH HumanEval+ MMLU-Pro sample efficiency cross-domain generalizability