title: SpecReason: Fast and Accurate Inference-Time Compute via Speculative
    Reasoning
summary: Recent advances in inference-time compute have significantly improved
  performance on complex tasks by generating long chains of thought (CoTs) using
  Large Reasoning Models (LRMs). However, this improved accuracy comes at the
  cost of high inference latency due to the length of generated reasoning
  sequences and the autoregressive nature of decoding. Our key insight in
  tackling these overheads is that LRM inference, and the reasoning that it
  embeds, is highly tolerant of approximations: complex tasks are typically
  broken down into simpler steps, each of which brings utility based on the
  semantic insight it provides for downstream steps rather than the exact tokens
  it generates. Accordingly, we introduce SpecReason, a system that automatically
  accelerates LRM inference by using a lightweight model to (speculatively) carry
  out simpler intermediate reasoning steps and reserving the costly base model
  only to assess (and potentially correct) the speculated outputs. Importantly,
  SpecReason's focus on exploiting the semantic flexibility of thinking tokens in
  preserving final-answer accuracy is complementary to prior speculation
  techniques, most notably speculative decoding, which demands token-level
  equivalence at each step. Across a variety of reasoning benchmarks, SpecReason
  achieves 1.5-2.5times speedup over vanilla LRM inference while improving
  accuracy by 1.0-9.9\%. Compared to speculative decoding without SpecReason,
  their combination yields an additional 19.4-44.2\% latency reduction. We
  open-source SpecReason at https://github.com/ruipeterpan/specreason.
paper_authors._id: 67fdaecd1c5d730135243f5c
paper_authors.name: Rui Pan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-15T07:55:03.136Z
paper_authors._id: 62221f7af14f39d27896de15
paper_authors.avatarUrl: /avatars/6318269df8c77e51f16445c29d65307c.svg
paper_authors.fullname: Rui Pan
paper_authors.type: user
paper_authors.user: ruipeterpan
paper_authors._id: 67fdaecd1c5d730135243f5d
paper_authors.name: Yinwei Dai
paper_authors._id: 67fdaecd1c5d730135243f5e
paper_authors.name: Zhihao Zhang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-04-15T00:56:48.883Z
paper_authors._id: 6480ec2d856901b0edbbd47a
paper_authors.avatarUrl: /avatars/dd8c22589c46fc19d7bf4a1329692d91.svg
paper_authors.fullname: Zhihao Jia
paper_authors.type: user
paper_authors.user: zhihaojia
paper_authors._id: 67fdaecd1c5d730135243f5f
paper_authors.name: Gabriele Oliaro
paper_authors._id: 67fdaecd1c5d730135243f60
paper_authors.name: Zhihao Jia
paper_authors._id: 67fdaecd1c5d730135243f61
paper_authors.name: Ravi Netravali
paper_title: SpecReason: Fast and Accurate Inference-Time Compute via Speculative
    Reasoning
paper_summary: Recent advances in inference-time compute have significantly improved
  performance on complex tasks by generating long chains of thought (CoTs) using
  Large Reasoning Models (LRMs). However, this improved accuracy comes at the
  cost of high inference latency due to the length of generated reasoning
  sequences and the autoregressive nature of decoding. Our key insight in
  tackling these overheads is that LRM inference, and the reasoning that it
  embeds, is highly tolerant of approximations: complex tasks are typically
  broken down into simpler steps, each of which brings utility based on the
  semantic insight it provides for downstream steps rather than the exact tokens
  it generates. Accordingly, we introduce SpecReason, a system that automatically
  accelerates LRM inference by using a lightweight model to (speculatively) carry
  out simpler intermediate reasoning steps and reserving the costly base model
  only to assess (and potentially correct) the speculated outputs. Importantly,
  SpecReason's focus on exploiting the semantic flexibility of thinking tokens in
  preserving final-answer accuracy is complementary to prior speculation
  techniques, most notably speculative decoding, which demands token-level
  equivalence at each step. Across a variety of reasoning benchmarks, SpecReason
  achieves 1.5-2.5times speedup over vanilla LRM inference while improving
  accuracy by 1.0-9.9\%. Compared to speculative decoding without SpecReason,
  their combination yields an additional 19.4-44.2\% latency reduction. We
  open-source SpecReason at https://github.com/ruipeterpan/specreason.
paper_ai_summary: SpecReason accelerates Large Reasoning Model inference by using a lightweight model for intermediate reasoning steps, improving speed without sacrificing accuracy.
paper_ai_keywords: SpecReason Large Reasoning Models inference latency reasoning sequences autoregressive decoding speculative decoding token-level equivalence