title: Small Models Struggle to Learn from Strong Reasoners
summary: Large language models (LLMs) excel in complex reasoning tasks, and distilling
  their reasoning capabilities into smaller models has shown promise. However, we
  uncover an interesting phenomenon, which we term the Small Model Learnability
  Gap: small models (leq3B parameters) do not consistently benefit from long
  chain-of-thought (CoT) reasoning or distillation from larger models. Instead,
  they perform better when fine-tuned on shorter, simpler reasoning chains that
  better align with their intrinsic learning capacity. To address this, we
  propose Mix Distillation, a simple yet effective strategy that balances
  reasoning complexity by combining long and short CoT examples or reasoning from
  both larger and smaller models. Our experiments demonstrate that Mix
  Distillation significantly improves small model reasoning performance compared
  to training on either data alone. These findings highlight the limitations of
  direct strong model distillation and underscore the importance of adapting
  reasoning complexity for effective reasoning capability transfer.
paper_authors._id: 67b4d05a9f8a8ab661450397
paper_authors.name: Yuetai Li
paper_authors._id: 67b4d05a9f8a8ab661450398
paper_authors.name: Xiang Yue
paper_authors._id: 67b4d05a9f8a8ab661450399
paper_authors.name: Zhangchen Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T09:37:32.715Z
paper_authors._id: 653df1323479e9ebbe3eb6cc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/653df1323479e9ebbe3eb6cc/K_g-r1iMRNKj99LXPuYF3.jpeg
paper_authors.fullname: Zhangchen Xu
paper_authors.type: user
paper_authors.user: zhangchenxu
paper_authors._id: 67b4d05a9f8a8ab66145039a
paper_authors.name: Fengqing Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:09:45.083Z
paper_authors._id: 6531e1021dd8ebbdc1a6fd8e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6531e1021dd8ebbdc1a6fd8e/lIcl7zCPtzRsfiUh6uY1o.jpeg
paper_authors.fullname: Fengqing Jiang
paper_authors.type: user
paper_authors.user: fqjiang
paper_authors._id: 67b4d05a9f8a8ab66145039b
paper_authors.name: Luyao Niu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:09:50.890Z
paper_authors._id: 666dfd4770f5a2cb4aefd12f
paper_authors.avatarUrl: /avatars/fa0e0dbc203a21e58dda8fdb4cbc67ad.svg
paper_authors.fullname: Luyao Niu
paper_authors.type: user
paper_authors.user: LNIU
paper_authors._id: 67b4d05a9f8a8ab66145039c
paper_authors.name: Bill Yuchen Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:09:56.614Z
paper_authors._id: 607f666a4ad99100d63ce35c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/607f666a4ad99100d63ce35c/QxhxnvfeV6efkxwUFHwjI.png
paper_authors.fullname: Bill Yuchen Lin
paper_authors.type: user
paper_authors.user: yuchenlin
paper_authors._id: 67b4d05a9f8a8ab66145039d
paper_authors.name: Bhaskar Ramasubramanian
paper_authors._id: 67b4d05a9f8a8ab66145039e
paper_authors.name: Radha Poovendran
paper_title: Small Models Struggle to Learn from Strong Reasoners
paper_summary: Large language models (LLMs) excel in complex reasoning tasks, and distilling
  their reasoning capabilities into smaller models has shown promise. However, we
  uncover an interesting phenomenon, which we term the Small Model Learnability
  Gap: small models (leq3B parameters) do not consistently benefit from long
  chain-of-thought (CoT) reasoning or distillation from larger models. Instead,
  they perform better when fine-tuned on shorter, simpler reasoning chains that
  better align with their intrinsic learning capacity. To address this, we
  propose Mix Distillation, a simple yet effective strategy that balances
  reasoning complexity by combining long and short CoT examples or reasoning from
  both larger and smaller models. Our experiments demonstrate that Mix
  Distillation significantly improves small model reasoning performance compared
  to training on either data alone. These findings highlight the limitations of
  direct strong model distillation and underscore the importance of adapting
  reasoning complexity for effective reasoning capability transfer.
paper_ai_keywords: chain-of-thought (CoT) reasoning chains intrinsic learning capacity Mix Distillation