title: AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via
    Reinforcement Learning
summary: Large Language Models (LLMs) have demonstrated remarkable capabilities but
  often face challenges with tasks requiring sophisticated reasoning. While
  Chain-of-Thought (CoT) prompting significantly enhances reasoning, it
  indiscriminately generates lengthy reasoning steps for all queries, leading to
  substantial computational costs and inefficiency, especially for simpler
  inputs. To address this critical issue, we introduce AdaCoT (Adaptive
  Chain-of-Thought), a novel framework enabling LLMs to adaptively decide when to
  invoke CoT. AdaCoT framed adaptive reasoning as a Pareto optimization problem
  that seeks to balance model performance with the costs associated with CoT
  invocation (both frequency and computational overhead). We propose a
  reinforcement learning (RL) based method, specifically utilizing Proximal
  Policy Optimization (PPO), to dynamically control the CoT triggering decision
  boundary by adjusting penalty coefficients, thereby allowing the model to
  determine CoT necessity based on implicit query complexity. A key technical
  contribution is Selective Loss Masking (SLM), designed to counteract decision
  boundary collapse during multi-stage RL training, ensuring robust and stable
  adaptive triggering. Experimental results demonstrate that AdaCoT successfully
  navigates the Pareto frontier, achieving substantial reductions in CoT usage
  for queries not requiring elaborate reasoning. For instance, on our production
  traffic testset, AdaCoT reduced CoT triggering rates to as low as 3.18\% and
  decreased average response tokens by 69.06%, while maintaining high performance
  on complex tasks.
paper_authors._id: 682bf60625f785dbadfb3dfd
paper_authors.name: Chenwei Lou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:12:03.700Z
paper_authors._id: 63fc6e47ee821f4bdfab58b8
paper_authors.avatarUrl: /avatars/4f1e98050092e416ba543b66dd981c2e.svg
paper_authors.fullname: louchenwei
paper_authors.type: user
paper_authors.user: louchenwei
paper_authors._id: 682bf60625f785dbadfb3dfe
paper_authors.name: Zewei Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:12:13.589Z
paper_authors._id: 638dbeaaf467129f49947d5b
paper_authors.avatarUrl: /avatars/996aa78b4edb429cbb436d48821a317b.svg
paper_authors.fullname: Zewei Sun
paper_authors.type: user
paper_authors.user: sunzewei2715
paper_authors._id: 682bf60625f785dbadfb3dff
paper_authors.name: Xinnian Liang
paper_authors._id: 682bf60625f785dbadfb3e00
paper_authors.name: Meng Qu
paper_authors._id: 682bf60625f785dbadfb3e01
paper_authors.name: Wei Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:11:52.139Z
paper_authors._id: 6468823272d9180d4ac90bdf
paper_authors.avatarUrl: /avatars/70cb7d65d30ecb944595000ceeeedb1b.svg
paper_authors.fullname: Wei Shen
paper_authors.type: user
paper_authors.user: Swtheking
paper_authors._id: 682bf60625f785dbadfb3e02
paper_authors.name: Wenqi Wang
paper_authors._id: 682bf60625f785dbadfb3e03
paper_authors.name: Yuntao Li
paper_authors._id: 682bf60625f785dbadfb3e04
paper_authors.name: Qingping Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:23:10.993Z
paper_authors._id: 64d20e1821aed29b2ffd2d99
paper_authors.avatarUrl: /avatars/b0719319a74e8f51fc8a1404aca367e6.svg
paper_authors.fullname: Qingping Yang
paper_authors.type: user
paper_authors.user: qingping95
paper_authors._id: 682bf60625f785dbadfb3e05
paper_authors.name: Shuangzhi Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:23:43.591Z
paper_authors._id: 637301f4bb66bd6b13206a25
paper_authors.avatarUrl: /avatars/6925439441324f6fd00d167d471edff2.svg
paper_authors.fullname: Shuangzhi Wu
paper_authors.type: user
paper_authors.user: Shuangzhi
paper_title: AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via
    Reinforcement Learning
paper_summary: Large Language Models (LLMs) have demonstrated remarkable capabilities but
  often face challenges with tasks requiring sophisticated reasoning. While
  Chain-of-Thought (CoT) prompting significantly enhances reasoning, it
  indiscriminately generates lengthy reasoning steps for all queries, leading to
  substantial computational costs and inefficiency, especially for simpler
  inputs. To address this critical issue, we introduce AdaCoT (Adaptive
  Chain-of-Thought), a novel framework enabling LLMs to adaptively decide when to
  invoke CoT. AdaCoT framed adaptive reasoning as a Pareto optimization problem
  that seeks to balance model performance with the costs associated with CoT
  invocation (both frequency and computational overhead). We propose a
  reinforcement learning (RL) based method, specifically utilizing Proximal
  Policy Optimization (PPO), to dynamically control the CoT triggering decision
  boundary by adjusting penalty coefficients, thereby allowing the model to
  determine CoT necessity based on implicit query complexity. A key technical
  contribution is Selective Loss Masking (SLM), designed to counteract decision
  boundary collapse during multi-stage RL training, ensuring robust and stable
  adaptive triggering. Experimental results demonstrate that AdaCoT successfully
  navigates the Pareto frontier, achieving substantial reductions in CoT usage
  for queries not requiring elaborate reasoning. For instance, on our production
  traffic testset, AdaCoT reduced CoT triggering rates to as low as 3.18\% and
  decreased average response tokens by 69.06%, while maintaining high performance
  on complex tasks.
paper_ai_summary: AdaCoT, an adaptive reasoning framework using reinforcement learning, reduces unnecessary Chain-of-Thought generation by LLMs, cutting computational costs without sacrificing performance on complex tasks.
paper_ai_keywords: Chain-of-Thought AdaCoT Pareto optimization reinforcement learning Proximal Policy Optimization Selective Loss Masking