AdaptThink: Reasoning Models Can Learn When to Think
Recently, large reasoning models have achieved impressive performance on
various tasks by employing human-like deep thinking. However, the lengthy
thinking process substantially increases inference overhead, making efficiency
a critical bottleneck. In this work, we first demonstrate that NoThinking,
which prompts the reasoning model to skip thinking and directly generate the
final solution, is a better choice for relatively simple tasks in terms of both
performance and efficiency. Motivated by this, we propose AdaptThink, a novel
RL algorithm to teach reasoning models to choose the optimal thinking mode
adaptively based on problem difficulty. Specifically, AdaptThink features two
core components: (1) a constrained optimization objective that encourages the
model to choose NoThinking while maintaining the overall performance; (2) an
importance sampling strategy that balances Thinking and NoThinking samples
during on-policy training, thereby enabling cold start and allowing the model
to explore and exploit both thinking modes throughout the training process. Our
experiments indicate that AdaptThink significantly reduces the inference costs
while further enhancing performance. Notably, on three math datasets,
AdaptThink reduces the average response length of DeepSeek-R1-Distill-Qwen-1.5B
by 53% and improves its accuracy by 2.4%, highlighting the promise of adaptive
thinking-mode selection for optimizing the balance between reasoning quality
and efficiency. Our codes and models are available at
https://github.com/THU-KEG/AdaptThink.
682be3e43ba4cfbca886a521
Jiajie Zhang
admin_assigned
2025-05-20T08:10:15.004Z
66cdd285c51a915bd5f2d017
/avatars/14e5794307e4672b1b51d26b31227e0f.svg
Jiajie Zhang
user
NeoZ123
682be3e43ba4cfbca886a522
Nianyi Lin
admin_assigned
2025-05-20T08:10:51.330Z
67385497d9af4eb4c078ced3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/yP-EPaY0tUosVR4kjXQ9B.png
Lin Nianyi
user
linny2002
682be3e43ba4cfbca886a523
Lei Hou
682be3e43ba4cfbca886a524
Ling Feng
682be3e43ba4cfbca886a525
Juanzi Li
admin_assigned
2025-05-20T08:10:58.673Z
65df8cbc2705d9672f55d1aa
/avatars/63e46f15bb76bd9d4508fd0f54f39829.svg
Juanzi Li
user
juanli
AdaptThink: Reasoning Models Can Learn When to Think
Recently, large reasoning models have achieved impressive performance on
various tasks by employing human-like deep thinking. However, the lengthy
thinking process substantially increases inference overhead, making efficiency
a critical bottleneck. In this work, we first demonstrate that NoThinking,
which prompts the reasoning model to skip thinking and directly generate the
final solution, is a better choice for relatively simple tasks in terms of both
performance and efficiency. Motivated by this, we propose AdaptThink, a novel
RL algorithm to teach reasoning models to choose the optimal thinking mode
adaptively based on problem difficulty. Specifically, AdaptThink features two
core components: (1) a constrained optimization objective that encourages the
model to choose NoThinking while maintaining the overall performance; (2) an
importance sampling strategy that balances Thinking and NoThinking samples
during on-policy training, thereby enabling cold start and allowing the model
to explore and exploit both thinking modes throughout the training process. Our
experiments indicate that AdaptThink significantly reduces the inference costs
while further enhancing performance. Notably, on three math datasets,
AdaptThink reduces the average response length of DeepSeek-R1-Distill-Qwen-1.5B
by 53% and improves its accuracy by 2.4%, highlighting the promise of adaptive
thinking-mode selection for optimizing the balance between reasoning quality
and efficiency. Our codes and models are available at
https://github.com/THU-KEG/AdaptThink.
AdaptThink, an RL algorithm, enhances reasoning models' efficiency and performance by adaptively selecting the optimal thinking mode based on problem difficulty.
NoThinking Reinforcement Learning (RL) constrained optimization objective importance sampling strategy on-policy training DeepSeek-R1-Distill-Qwen-1.5B reasoning quality efficiency