title: Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language
    Models in Math
summary: Chain-of-Thought (CoT) significantly enhances formal reasoning capabilities
  in Large Language Models (LLMs) by training them to explicitly generate
  intermediate reasoning steps. While LLMs readily benefit from such techniques,
  improving reasoning in Small Language Models (SLMs) remains challenging due to
  their limited model capacity. Recent work by Deepseek-R1 demonstrates that
  distillation from LLM-generated synthetic data can substantially improve the
  reasoning ability of SLM. However, the detailed modeling recipe is not
  disclosed. In this work, we present a systematic training recipe for SLMs that
  consists of four steps: (1) large-scale mid-training on diverse distilled
  long-CoT data, (2) supervised fine-tuning on high-quality long-CoT data, (3)
  Rollout DPO leveraging a carefully curated preference dataset, and (4)
  Reinforcement Learning (RL) with Verifiable Reward. We apply our method on
  Phi-4-Mini, a compact 3.8B-parameter model. The resulting Phi-4-Mini-Reasoning
  model exceeds, on math reasoning tasks, much larger reasoning models, e.g.,
  outperforming DeepSeek-R1-Distill-Qwen-7B by 3.2 points and
  DeepSeek-R1-Distill-Llama-8B by 7.7 points on Math-500. Our results validate
  that a carefully designed training recipe, with large-scale high-quality CoT
  data, is effective to unlock strong reasoning capabilities even in
  resource-constrained small models.
paper_authors._id: 6812d62ae74b39182bd17c9c
paper_authors.name: Haoran Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:53:11.914Z
paper_authors._id: 61384b860317b0a5c10877d3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631080954171-61384b860317b0a5c10877d3.jpeg
paper_authors.fullname: Haoran Xu
paper_authors.type: user
paper_authors.user: haoranxu
paper_authors._id: 6812d62ae74b39182bd17c9d
paper_authors.name: Baolin Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:53:31.380Z
paper_authors._id: 61942296d5c2ba6daa290357
paper_authors.avatarUrl: /avatars/594021cc183c4922d48b46f43772a062.svg
paper_authors.fullname: Baolin Peng
paper_authors.type: user
paper_authors.user: Baolin
paper_authors._id: 6812d62ae74b39182bd17c9e
paper_authors.name: Hany Awadalla
paper_authors._id: 6812d62ae74b39182bd17c9f
paper_authors.name: Dongdong Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:53:43.028Z
paper_authors._id: 6567651c6fcc82e5e8c36d4d
paper_authors.avatarUrl: /avatars/ba3cc037a7688c4f8d967fc6043e540d.svg
paper_authors.fullname: Dongdong Chen
paper_authors.type: user
paper_authors.user: dongdongchen
paper_authors._id: 6812d62ae74b39182bd17ca0
paper_authors.name: Yen-Chun Chen
paper_authors._id: 6812d62ae74b39182bd17ca1
paper_authors.name: Mei Gao
paper_authors._id: 6812d62ae74b39182bd17ca2
paper_authors.name: Young Jin Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-02T06:35:02.393Z
paper_authors._id: 63f5173bb51da4d61da6c038
paper_authors.avatarUrl: /avatars/0ee530cf80476aa3985c4d591cd384a1.svg
paper_authors.fullname: Young Jin Kim
paper_authors.type: user
paper_authors.user: ykim362
paper_authors._id: 6812d62ae74b39182bd17ca3
paper_authors.name: Yunsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:15.501Z
paper_authors._id: 65b9a5a498bd7a573bb98408
paper_authors.avatarUrl: /avatars/efab4e24c36db6a5d50263677308cb8a.svg
paper_authors.fullname: Yunsheng Li
paper_authors.type: user
paper_authors.user: liyunsheng13
paper_authors._id: 6812d62ae74b39182bd17ca4
paper_authors.name: Liliang Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:21.232Z
paper_authors._id: 63815eff4761ddfa00903762
paper_authors.avatarUrl: /avatars/3419b239d42e091586f1c51b526d88e5.svg
paper_authors.fullname: Liliang Ren
paper_authors.type: user
paper_authors.user: renll
paper_authors._id: 6812d62ae74b39182bd17ca5
paper_authors.name: Yelong Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:30.183Z
paper_authors._id: 6454c337a13edf669cd5d8ea
paper_authors.avatarUrl: /avatars/a383a0dda7c2ef6a0d6c3c64651f42ff.svg
paper_authors.fullname: Yelong Shen
paper_authors.type: user
paper_authors.user: uuu6
paper_authors._id: 6812d62ae74b39182bd17ca6
paper_authors.name: Shuohang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:38.929Z
paper_authors._id: 6463b2247572c66a8e625a57
paper_authors.avatarUrl: /avatars/7722fb5649d42d966ce1e478946d5f8f.svg
paper_authors.fullname: Wang
paper_authors.type: user
paper_authors.user: Shuohang
paper_authors._id: 6812d62ae74b39182bd17ca7
paper_authors.name: Weijian Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:44.400Z
paper_authors._id: 6398f4b32c20654083f36cde
paper_authors.avatarUrl: /avatars/4591f514483890997c55e9e6d60bbb0f.svg
paper_authors.fullname: Weijian Xu
paper_authors.type: user
paper_authors.user: xwjabc
paper_authors._id: 6812d62ae74b39182bd17ca8
paper_authors.name: Jianfeng Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:50.324Z
paper_authors._id: 641904caf9d6f1d772ec7af7
paper_authors.avatarUrl: /avatars/4a63eac71eb30f70b1a0e9d4708f26c1.svg
paper_authors.fullname: Jianfeng Gao
paper_authors.type: user
paper_authors.user: wyngjf
paper_authors._id: 6812d62ae74b39182bd17ca9
paper_authors.name: Weizhu Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:54:56.808Z
paper_authors._id: 64da876370446182be5b608d
paper_authors.avatarUrl: /avatars/e412fdc71404ecdf638e416846e3ebfb.svg
paper_authors.fullname: Weizhu Chen
paper_authors.type: user
paper_authors.user: chenweizhu
paper_title: Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language
    Models in Math
paper_summary: Chain-of-Thought (CoT) significantly enhances formal reasoning capabilities
  in Large Language Models (LLMs) by training them to explicitly generate
  intermediate reasoning steps. While LLMs readily benefit from such techniques,
  improving reasoning in Small Language Models (SLMs) remains challenging due to
  their limited model capacity. Recent work by Deepseek-R1 demonstrates that
  distillation from LLM-generated synthetic data can substantially improve the
  reasoning ability of SLM. However, the detailed modeling recipe is not
  disclosed. In this work, we present a systematic training recipe for SLMs that
  consists of four steps: (1) large-scale mid-training on diverse distilled
  long-CoT data, (2) supervised fine-tuning on high-quality long-CoT data, (3)
  Rollout DPO leveraging a carefully curated preference dataset, and (4)
  Reinforcement Learning (RL) with Verifiable Reward. We apply our method on
  Phi-4-Mini, a compact 3.8B-parameter model. The resulting Phi-4-Mini-Reasoning
  model exceeds, on math reasoning tasks, much larger reasoning models, e.g.,
  outperforming DeepSeek-R1-Distill-Qwen-7B by 3.2 points and
  DeepSeek-R1-Distill-Llama-8B by 7.7 points on Math-500. Our results validate
  that a carefully designed training recipe, with large-scale high-quality CoT
  data, is effective to unlock strong reasoning capabilities even in
  resource-constrained small models.
paper_ai_summary: A structured training method with high-quality Chain-of-Thought data improves reasoning abilities in small language models.
paper_ai_keywords: Chain-of-Thought Large Language Models Small Language Models distillation large-scale mid-training supervised fine-tuning Rollout DPO Reinforcement Learning Verifiable Reward math reasoning tasks