title: Compressed Chain of Thought: Efficient Reasoning Through Dense
    Representations
summary: Chain-of-thought (CoT) decoding enables language models to improve reasoning
  performance at the cost of high generation latency in decoding. Recent
  proposals have explored variants of contemplation tokens, a term we introduce
  that refers to special tokens used during inference to allow for extra
  computation. Prior work has considered fixed-length sequences drawn from a
  discrete set of embeddings as contemplation tokens. Here we propose Compressed
  Chain-of-Thought (CCoT), a framework to generate contentful and continuous
  contemplation tokens of variable sequence length. The generated contemplation
  tokens are compressed representations of explicit reasoning chains, and our
  method can be applied to off-the-shelf decoder language models. Through
  experiments, we illustrate how CCoT enables additional reasoning over dense
  contentful representations to achieve corresponding improvements in accuracy.
  Moreover, the reasoning improvements can be adaptively modified on demand by
  controlling the number of contemplation tokens generated.
paper_authors._id: 6762f21ea3eed649f9889a1c
paper_authors.name: Jeffrey Cheng
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-20T15:37:24.321Z
paper_authors._id: 65f28eebf5cf26fe0632ce67
paper_authors.avatarUrl: /avatars/cd970cfe4215374c82d47df57ac30795.svg
paper_authors.fullname: Jeffrey Cheng
paper_authors.type: user
paper_authors.user: nexync
paper_authors._id: 6762f21ea3eed649f9889a1d
paper_authors.name: Benjamin Van Durme
paper_title: Compressed Chain of Thought: Efficient Reasoning Through Dense
    Representations
paper_summary: Chain-of-thought (CoT) decoding enables language models to improve reasoning
  performance at the cost of high generation latency in decoding. Recent
  proposals have explored variants of contemplation tokens, a term we introduce
  that refers to special tokens used during inference to allow for extra
  computation. Prior work has considered fixed-length sequences drawn from a
  discrete set of embeddings as contemplation tokens. Here we propose Compressed
  Chain-of-Thought (CCoT), a framework to generate contentful and continuous
  contemplation tokens of variable sequence length. The generated contemplation
  tokens are compressed representations of explicit reasoning chains, and our
  method can be applied to off-the-shelf decoder language models. Through
  experiments, we illustrate how CCoT enables additional reasoning over dense
  contentful representations to achieve corresponding improvements in accuracy.
  Moreover, the reasoning improvements can be adaptively modified on demand by
  controlling the number of contemplation tokens generated.
paper_ai_summary: Compressed Chain-of-Thought (CCoT) enhances language model reasoning by generating efficient, variable-length contemplation tokens during inference, improving accuracy without fixed-length tokens.
paper_ai_keywords: Chain-of-thought decoding contemplation tokens discrete set of embeddings continuous contemplation tokens variable sequence length explicit reasoning chains off-the-shelf decoder language models reasoning accuracy