The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a
  Reasoning Model will Think
Long chain-of-thought (CoT) is an essential ingredient in effective usage of
modern large language models, but our understanding of the reasoning strategies
underlying these capabilities remains limited. While some prior works have
attempted to categorize CoTs using predefined strategy types, such approaches
are constrained by human intuition and fail to capture the full diversity of
model behaviors. In this work, we introduce the CoT Encyclopedia, a bottom-up
framework for analyzing and steering model reasoning. Our method automatically
extracts diverse reasoning criteria from model-generated CoTs, embeds them into
a semantic space, clusters them into representative categories, and derives
contrastive rubrics to interpret reasoning behavior. Human evaluations show
that this framework produces more interpretable and comprehensive analyses than
existing methods. Moreover, we demonstrate that this understanding enables
performance gains: we can predict which strategy a model is likely to use and
guide it toward more effective alternatives. Finally, we provide practical
insights, such as that training data format (e.g., free-form vs.
multiple-choice) has a far greater impact on reasoning behavior than data
domain, underscoring the importance of format-aware model design.
68269f67a47cb2b87646b98c
Seongyun Lee
admin_assigned
2025-05-16T08:14:03.612Z
6550c4f27bbfce1878f5f280
/avatars/0ecedbcd8a55b2c4abd1da9e741a6652.svg
seongyun_lee
user
Seongyun
68269f67a47cb2b87646b98d
Seungone Kim
admin_assigned
2025-05-16T08:14:19.025Z
6469949654873f0043b09c22
https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/Lk7IJAR16Wa_sGJ2g81AQ.jpeg
Seungone Kim
user
seungone
68269f67a47cb2b87646b98e
Minju Seo
admin_assigned
2025-05-16T08:14:25.204Z
66f10ac605182775917d8c5a
/avatars/21b21284d0a5a95413f91dde9dda346c.svg
Minju Seo
user
Minju2136
68269f67a47cb2b87646b98f
Yongrae Jo
admin_assigned
2025-05-16T08:14:30.806Z
649e06313e5e7504763dfe03
/avatars/1c4d19de5f2950d3342480c4b3e01047.svg
Yongrae Jo
user
dreamgonfly
68269f67a47cb2b87646b990
Dongyoung Go
68269f67a47cb2b87646b991
Hyeonbin Hwang
admin_assigned
2025-05-16T08:14:44.935Z
647eaaf61a1fcad2fdc5d1ef
/avatars/596d7a61d6e6227d38b661210a32fed0.svg
Hyeonbin Hwang 
user
hbin0701
68269f67a47cb2b87646b992
Jinho Park
admin_assigned
2025-05-16T08:14:51.443Z
638467ee8283412d401770dd
https://cdn-avatars.huggingface.co/v1/production/uploads/638467ee8283412d401770dd/UAMwDHhSwf91XSsubfrS_.jpeg
Jinho Park
user
Br3ad
68269f67a47cb2b87646b993
Xiang Yue
claimed_verified
2025-07-02T16:08:06.173Z
6230d750d93e84e233882dbc
https://cdn-avatars.huggingface.co/v1/production/uploads/6230d750d93e84e233882dbc/4MGEekLW3oWzqeFWDWvIK.jpeg
Xiang Yue
user
yuexiang96
68269f67a47cb2b87646b994
Sean Welleck
admin_assigned
2025-05-16T08:15:17.990Z
63e3f3c59db5da2dc1ef6889
/avatars/f7546f57a5fd69bc99ff1640cc4a4853.svg
Sean Welleck
user
wellecks
68269f67a47cb2b87646b995
Graham Neubig
admin_assigned
2025-05-16T08:15:24.404Z
60de14638bedd2315529d43f
https://cdn-avatars.huggingface.co/v1/production/uploads/1625166923504-noauth.png
Graham Neubig
user
gneubig
68269f67a47cb2b87646b996
Moontae Lee
68269f67a47cb2b87646b997
Minjoon Seo
admin_assigned
2025-05-16T08:15:33.521Z
621f05ba970615ad5861ceb1
/avatars/7e1902aa71369a524afda9b0a9e88e22.svg
Minjoon Seo
user
minjoon
The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a
  Reasoning Model will Think
Long chain-of-thought (CoT) is an essential ingredient in effective usage of
modern large language models, but our understanding of the reasoning strategies
underlying these capabilities remains limited. While some prior works have
attempted to categorize CoTs using predefined strategy types, such approaches
are constrained by human intuition and fail to capture the full diversity of
model behaviors. In this work, we introduce the CoT Encyclopedia, a bottom-up
framework for analyzing and steering model reasoning. Our method automatically
extracts diverse reasoning criteria from model-generated CoTs, embeds them into
a semantic space, clusters them into representative categories, and derives
contrastive rubrics to interpret reasoning behavior. Human evaluations show
that this framework produces more interpretable and comprehensive analyses than
existing methods. Moreover, we demonstrate that this understanding enables
performance gains: we can predict which strategy a model is likely to use and
guide it toward more effective alternatives. Finally, we provide practical
insights, such as that training data format (e.g., free-form vs.
multiple-choice) has a far greater impact on reasoning behavior than data
domain, underscoring the importance of format-aware model design.
The CoT Encyclopedia is a bottom-up framework that automatically analyzes and categorizes reasoning strategies in long chain-of-thoughts generated by large language models, providing more interpretable analyses and enabling performance improvements.
long chain-of-thought modern large language models reasoning strategies CoT Encyclopedia semantic space clusters contrastive rubrics free-form vs. multiple-choice format-aware model design