AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via
  Reinforcement Learning
Large Language Models (LLMs) have demonstrated remarkable capabilities but
often face challenges with tasks requiring sophisticated reasoning. While
Chain-of-Thought (CoT) prompting significantly enhances reasoning, it
indiscriminately generates lengthy reasoning steps for all queries, leading to
substantial computational costs and inefficiency, especially for simpler
inputs. To address this critical issue, we introduce AdaCoT (Adaptive
Chain-of-Thought), a novel framework enabling LLMs to adaptively decide when to
invoke CoT. AdaCoT framed adaptive reasoning as a Pareto optimization problem
that seeks to balance model performance with the costs associated with CoT
invocation (both frequency and computational overhead). We propose a
reinforcement learning (RL) based method, specifically utilizing Proximal
Policy Optimization (PPO), to dynamically control the CoT triggering decision
boundary by adjusting penalty coefficients, thereby allowing the model to
determine CoT necessity based on implicit query complexity. A key technical
contribution is Selective Loss Masking (SLM), designed to counteract decision
boundary collapse during multi-stage RL training, ensuring robust and stable
adaptive triggering. Experimental results demonstrate that AdaCoT successfully
navigates the Pareto frontier, achieving substantial reductions in CoT usage
for queries not requiring elaborate reasoning. For instance, on our production
traffic testset, AdaCoT reduced CoT triggering rates to as low as 3.18\% and
decreased average response tokens by 69.06%, while maintaining high performance
on complex tasks.
682bf60625f785dbadfb3dfd
Chenwei Lou
admin_assigned
2025-05-20T08:12:03.700Z
63fc6e47ee821f4bdfab58b8
/avatars/4f1e98050092e416ba543b66dd981c2e.svg
louchenwei
user
louchenwei
682bf60625f785dbadfb3dfe
Zewei Sun
admin_assigned
2025-05-20T08:12:13.589Z
638dbeaaf467129f49947d5b
/avatars/996aa78b4edb429cbb436d48821a317b.svg
Zewei Sun
user
sunzewei2715
682bf60625f785dbadfb3dff
Xinnian Liang
claimed_verified
2025-06-19T09:10:52.135Z
60adea21e3de7c7440abb863
/avatars/7fdba0e15ea435a4ea41106df574f312.svg
Xinnian Liang
user
EricLiang98
682bf60625f785dbadfb3e00
Meng Qu
682bf60625f785dbadfb3e01
Wei Shen
admin_assigned
2025-05-20T08:11:52.139Z
6468823272d9180d4ac90bdf
/avatars/70cb7d65d30ecb944595000ceeeedb1b.svg
Wei Shen
user
Swtheking
682bf60625f785dbadfb3e02
Wenqi Wang
682bf60625f785dbadfb3e03
Yuntao Li
682bf60625f785dbadfb3e04
Qingping Yang
admin_assigned
2025-05-20T08:23:10.993Z
64d20e1821aed29b2ffd2d99
/avatars/b0719319a74e8f51fc8a1404aca367e6.svg
Qingping Yang
user
qingping95
682bf60625f785dbadfb3e05
Shuangzhi Wu
admin_assigned
2025-05-20T08:23:43.591Z
637301f4bb66bd6b13206a25
/avatars/6925439441324f6fd00d167d471edff2.svg
Shuangzhi Wu
user
Shuangzhi
AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via
  Reinforcement Learning
Large Language Models (LLMs) have demonstrated remarkable capabilities but
often face challenges with tasks requiring sophisticated reasoning. While
Chain-of-Thought (CoT) prompting significantly enhances reasoning, it
indiscriminately generates lengthy reasoning steps for all queries, leading to
substantial computational costs and inefficiency, especially for simpler
inputs. To address this critical issue, we introduce AdaCoT (Adaptive
Chain-of-Thought), a novel framework enabling LLMs to adaptively decide when to
invoke CoT. AdaCoT framed adaptive reasoning as a Pareto optimization problem
that seeks to balance model performance with the costs associated with CoT
invocation (both frequency and computational overhead). We propose a
reinforcement learning (RL) based method, specifically utilizing Proximal
Policy Optimization (PPO), to dynamically control the CoT triggering decision
boundary by adjusting penalty coefficients, thereby allowing the model to
determine CoT necessity based on implicit query complexity. A key technical
contribution is Selective Loss Masking (SLM), designed to counteract decision
boundary collapse during multi-stage RL training, ensuring robust and stable
adaptive triggering. Experimental results demonstrate that AdaCoT successfully
navigates the Pareto frontier, achieving substantial reductions in CoT usage
for queries not requiring elaborate reasoning. For instance, on our production
traffic testset, AdaCoT reduced CoT triggering rates to as low as 3.18\% and
decreased average response tokens by 69.06%, while maintaining high performance
on complex tasks.
AdaCoT, an adaptive reasoning framework using reinforcement learning, reduces unnecessary Chain-of-Thought generation by LLMs, cutting computational costs without sacrificing performance on complex tasks.
Chain-of-Thought AdaCoT Pareto optimization reinforcement learning Proximal Policy Optimization Selective Loss Masking