title: R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large
    Model Token Routing
summary: Large Language Models (LLMs) achieve impressive reasoning capabilities at the
  cost of substantial inference overhead, posing substantial deployment
  challenges. Although distilled Small Language Models (SLMs) significantly
  enhance efficiency, their performance suffers as they fail to follow LLMs'
  reasoning paths. Luckily, we reveal that only a small fraction of tokens
  genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens
  are either identical or exhibit neutral differences, such as minor variations
  in abbreviations or expressions. Leveraging this insight, we introduce **Roads
  to Rome (R2R)**, a neural token routing method that selectively utilizes LLMs
  only for these critical, path-divergent tokens, while leaving the majority of
  token generation to the SLM. We also develop an automatic data generation
  pipeline that identifies divergent tokens and generates token-level routing
  labels to train the lightweight router. We apply R2R to combine R1-1.5B and
  R1-32B models from the DeepSeek family, and evaluate on challenging math,
  coding, and QA benchmarks. With an average activated parameter size of 5.6B,
  R2R surpasses the average accuracy of R1-7B by 1.6x, outperforming even the
  R1-14B model. Compared to R1-32B, it delivers a 2.8x wall-clock speedup with
  comparable performance, advancing the Pareto frontier of test-time scaling
  efficiency. Our code is available at https://github.com/thu-nics/R2R.
paper_authors._id: 6837bc9c9937bcb69885799c
paper_authors.name: Tianyu Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:42:43.123Z
paper_authors._id: 6445fd9ba56444c355dcbcba
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6445fd9ba56444c355dcbcba/NCyRCD-MK-yA0_qY6I2y0.png
paper_authors.fullname: Tianyu Fu
paper_authors.type: user
paper_authors.user: fuvty
paper_authors._id: 6837bc9c9937bcb69885799d
paper_authors.name: Yi Ge
paper_authors._id: 6837bc9c9937bcb69885799e
paper_authors.name: Yichen You
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:42:40.790Z
paper_authors._id: 66954ebfbcd81f395e9dca37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66954ebfbcd81f395e9dca37/0C3m5YdxyXuK7dJBu4AdL.png
paper_authors.fullname: Yichen You
paper_authors.type: user
paper_authors.user: youyc22
paper_authors._id: 6837bc9c9937bcb69885799f
paper_authors.name: Enshu Liu
paper_authors._id: 6837bc9c9937bcb6988579a0
paper_authors.name: Zhihang Yuan
paper_authors._id: 6837bc9c9937bcb6988579a1
paper_authors.name: Guohao Dai
paper_authors._id: 6837bc9c9937bcb6988579a2
paper_authors.name: Shengen Yan
paper_authors._id: 6837bc9c9937bcb6988579a3
paper_authors.name: Huazhong Yang
paper_authors._id: 6837bc9c9937bcb6988579a4
paper_authors.name: Yu Wang
paper_title: R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large
    Model Token Routing
paper_summary: Large Language Models (LLMs) achieve impressive reasoning capabilities at the
  cost of substantial inference overhead, posing substantial deployment
  challenges. Although distilled Small Language Models (SLMs) significantly
  enhance efficiency, their performance suffers as they fail to follow LLMs'
  reasoning paths. Luckily, we reveal that only a small fraction of tokens
  genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens
  are either identical or exhibit neutral differences, such as minor variations
  in abbreviations or expressions. Leveraging this insight, we introduce **Roads
  to Rome (R2R)**, a neural token routing method that selectively utilizes LLMs
  only for these critical, path-divergent tokens, while leaving the majority of
  token generation to the SLM. We also develop an automatic data generation
  pipeline that identifies divergent tokens and generates token-level routing
  labels to train the lightweight router. We apply R2R to combine R1-1.5B and
  R1-32B models from the DeepSeek family, and evaluate on challenging math,
  coding, and QA benchmarks. With an average activated parameter size of 5.6B,
  R2R surpasses the average accuracy of R1-7B by 1.6x, outperforming even the
  R1-14B model. Compared to R1-32B, it delivers a 2.8x wall-clock speedup with
  comparable performance, advancing the Pareto frontier of test-time scaling
  efficiency. Our code is available at https://github.com/thu-nics/R2R.
paper_ai_summary: Roads to Rome (R2R) selectively utilizes large language models for critical reasoning tasks to enhance efficiency and performance in lightweight models.
paper_ai_keywords: Large Language Models (LLMs) Small Language Models (SLMs) token routing neural token routing token divergence token generation automatic data generation pipeline token-level routing labels R1-1.5B R1-32B math benchmarks coding benchmarks QA benchmarks parameter size activated parameters test-time scaling efficiency pareto frontier