Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable
abilities in complex reasoning tasks by scaling test-time compute and
exhibiting human-like deep thinking. However, we identify a phenomenon we term
underthinking, where o1-like LLMs frequently switch between different reasoning
thoughts without sufficiently exploring promising paths to reach a correct
solution. This behavior leads to inadequate depth of reasoning and decreased
performance, particularly on challenging mathematical problems. To
systematically analyze this issue, we conduct experiments on three challenging
test sets and two representative open-source o1-like models, revealing that
frequent thought switching correlates with incorrect responses. We introduce a
novel metric to quantify underthinking by measuring token efficiency in
incorrect answers. To address underthinking, we propose a decoding strategy
with thought switching penalty TIP that discourages premature transitions
between thoughts, encouraging deeper exploration of each reasoning path.
Experimental results demonstrate that our approach improves accuracy across
challenging datasets without requiring model fine-tuning. Our findings
contribute to understanding reasoning inefficiencies in o1-like LLMs and offer
a practical solution to enhance their problem-solving capabilities.
679c5ca666c379e215bc9e74
Yue Wang
679c5ca666c379e215bc9e75
Qiuzhi Liu
admin_assigned
2025-01-31T08:45:37.562Z
63e60ff62d704152abac8af8
/avatars/a54c34fb87a7ed5aeba792852747de92.svg
Qiuzhi Liu
user
Dennis364
679c5ca666c379e215bc9e76
Jiahao Xu
admin_assigned
2025-01-31T08:45:31.807Z
660399710f1fc2f16de18072
/avatars/c22a749cc45db693c2d9ea877c7cace4.svg
Jiahao Xu
user
Jiahao004
679c5ca666c379e215bc9e77
Tian Liang
679c5ca666c379e215bc9e78
Xingyu Chen
679c5ca666c379e215bc9e79
Zhiwei He
admin_assigned
2025-01-31T08:44:45.300Z
638439ca834d3558a398d035
https://cdn-avatars.huggingface.co/v1/production/uploads/1669609868550-noauth.png
Zhiwei He
user
zwhe99
679c5ca666c379e215bc9e7a
Linfeng Song
admin_assigned
2025-01-31T08:44:29.221Z
64c94eddcb2f1bf0e7db5a4d
/avatars/f7e2532d3c85d5e5b5a02c579ea68c3a.svg
Linfeng Song
user
freesunshine0316
679c5ca666c379e215bc9e7b
Dian Yu
admin_assigned
2025-01-31T08:44:23.114Z
62d58fd53bf5e059f7cc3245
/avatars/7a4f3ee4a37245f67efd26749d66a706.svg
Dian Yu
user
yudian
679c5ca666c379e215bc9e7c
Juntao Li
admin_assigned
2025-01-31T08:44:12.069Z
6670e285b0c03c4e9d6e0985
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/uCZHm4gKSHZ2b0hpHWgZv.jpeg
Juntao Li
user
douvleplus
679c5ca666c379e215bc9e7d
Zhuosheng Zhang
admin_assigned
2025-01-31T08:44:05.749Z
5f82f9f7f0801648bf8844b2
https://cdn-avatars.huggingface.co/v1/production/uploads/1669627733134-5f82f9f7f0801648bf8844b2.jpeg
Zhuosheng Zhang
user
cooelf
679c5ca666c379e215bc9e7e
Rui Wang
679c5ca666c379e215bc9e7f
Zhaopeng Tu
admin_assigned
2025-01-31T08:43:27.683Z
67485743561b1e6f9579389f
/avatars/8a4cc63bd7be388010bc329bb74582a1.svg
Zhaopeng Tu
user
zptu
679c5ca666c379e215bc9e80
Haitao Mi
admin_assigned
2025-01-31T08:43:21.871Z
65147a1426fbd558dbd08f1b
/avatars/86574ee2d5c22e940be1c4e50be88675.svg
Haitao Mi
user
haitaominlp
679c5ca666c379e215bc9e81
Dong Yu
Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable
abilities in complex reasoning tasks by scaling test-time compute and
exhibiting human-like deep thinking. However, we identify a phenomenon we term
underthinking, where o1-like LLMs frequently switch between different reasoning
thoughts without sufficiently exploring promising paths to reach a correct
solution. This behavior leads to inadequate depth of reasoning and decreased
performance, particularly on challenging mathematical problems. To
systematically analyze this issue, we conduct experiments on three challenging
test sets and two representative open-source o1-like models, revealing that
frequent thought switching correlates with incorrect responses. We introduce a
novel metric to quantify underthinking by measuring token efficiency in
incorrect answers. To address underthinking, we propose a decoding strategy
with thought switching penalty TIP that discourages premature transitions
between thoughts, encouraging deeper exploration of each reasoning path.
Experimental results demonstrate that our approach improves accuracy across
challenging datasets without requiring model fine-tuning. Our findings
contribute to understanding reasoning inefficiencies in o1-like LLMs and offer
a practical solution to enhance their problem-solving capabilities.
A new decoding strategy addresses underthinking in large language models by penalizing frequent thought switching, improving accuracy without fine-tuning.
large language models OpenAI o1 reasoning tasks underthinking thought switching token efficiency decoding strategy thought switching penalty problem-solving capabilities