title: CoT-Valve: Length-Compressible Chain-of-Thought Tuning
summary: Chain-of-Thought significantly enhances a model's reasoning capability, but
  it also comes with a considerable increase in inference costs due to long
  chains. With the observation that the reasoning path can be easily compressed
  under easy tasks but struggle on hard tasks, we explore the feasibility of
  elastically controlling the length of reasoning paths with only one model,
  thereby reducing the inference overhead of reasoning models dynamically based
  on task difficulty. We introduce a new tuning and inference strategy named
  CoT-Valve, designed to allow models to generate reasoning chains of varying
  lengths. To achieve this, we propose to identify a direction in the parameter
  space that, when manipulated, can effectively control the length of generated
  CoT. Moreover, we show that this property is valuable for compressing the
  reasoning chain. We construct datasets with chains from long to short for the
  same questions and explore two enhanced strategies for CoT-Valve: (1) a precise
  length-compressible CoT tuning method, and (2) a progressive chain length
  compression approach. Our experiments show that CoT-Valve successfully enables
  controllability and compressibility of the chain and shows better performance
  than the prompt-based control. We applied this method to QwQ-32B-Preview,
  reducing reasoning chains on GSM8K from 741 to 225 tokens with a minor
  performance drop (95.07% to 94.92%) and on AIME from 6827 to 4629 tokens, with
  only one additional incorrect answer.
paper_authors._id: 67aed173e6952709b47c0c5c
paper_authors.name: Xinyin Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:50:50.673Z
paper_authors._id: 64396ebc21221ac7411852b3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64396ebc21221ac7411852b3/SR0dC8N0bdj9tZFxYPpSf.jpeg
paper_authors.fullname: Xinyin Ma
paper_authors.type: user
paper_authors.user: horseee
paper_authors._id: 67aed173e6952709b47c0c5d
paper_authors.name: Guangnian Wan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:51:10.430Z
paper_authors._id: 6627cccfded9b7936d5d1d21
paper_authors.avatarUrl: /avatars/9216bdd9ed10c226f2d14edce4a10daa.svg
paper_authors.fullname: Guangnian Wan
paper_authors.type: user
paper_authors.user: bigglesworthnotcat
paper_authors._id: 67aed173e6952709b47c0c5e
paper_authors.name: Runpeng Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:51:16.707Z
paper_authors._id: 635364b3c41f548fe39db945
paper_authors.avatarUrl: /avatars/ad1916bbfabca0b6651c8eabacc5eba8.svg
paper_authors.fullname: Runpeng Yu
paper_authors.type: user
paper_authors.user: rp-yu
paper_authors._id: 67aed173e6952709b47c0c5f
paper_authors.name: Gongfan Fang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:51:22.345Z
paper_authors._id: 646a1939c37ca1e12308fe81
paper_authors.avatarUrl: /avatars/752e9d86018e7d33ad8bcd741203fd86.svg
paper_authors.fullname: Gongfan Fang
paper_authors.type: user
paper_authors.user: Vinnnf
paper_authors._id: 67aed173e6952709b47c0c60
paper_authors.name: Xinchao Wang
paper_title: CoT-Valve: Length-Compressible Chain-of-Thought Tuning
paper_summary: Chain-of-Thought significantly enhances a model's reasoning capability, but
  it also comes with a considerable increase in inference costs due to long
  chains. With the observation that the reasoning path can be easily compressed
  under easy tasks but struggle on hard tasks, we explore the feasibility of
  elastically controlling the length of reasoning paths with only one model,
  thereby reducing the inference overhead of reasoning models dynamically based
  on task difficulty. We introduce a new tuning and inference strategy named
  CoT-Valve, designed to allow models to generate reasoning chains of varying
  lengths. To achieve this, we propose to identify a direction in the parameter
  space that, when manipulated, can effectively control the length of generated
  CoT. Moreover, we show that this property is valuable for compressing the
  reasoning chain. We construct datasets with chains from long to short for the
  same questions and explore two enhanced strategies for CoT-Valve: (1) a precise
  length-compressible CoT tuning method, and (2) a progressive chain length
  compression approach. Our experiments show that CoT-Valve successfully enables
  controllability and compressibility of the chain and shows better performance
  than the prompt-based control. We applied this method to QwQ-32B-Preview,
  reducing reasoning chains on GSM8K from 741 to 225 tokens with a minor
  performance drop (95.07% to 94.92%) and on AIME from 6827 to 4629 tokens, with
  only one additional incorrect answer.
paper_ai_keywords: Chain-of-Thought (CoT) reasoning capability inference costs reasoning paths task dynamically CoT-Valve parameter space reasoning chains length-compressible CoT tuning method progressive chain length compression approach prompt-based control QWQ-32B-Preview GSM8K AIME