R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large
  Model Token Routing
Large Language Models (LLMs) achieve impressive reasoning capabilities at the
cost of substantial inference overhead, posing substantial deployment
challenges. Although distilled Small Language Models (SLMs) significantly
enhance efficiency, their performance suffers as they fail to follow LLMs'
reasoning paths. Luckily, we reveal that only a small fraction of tokens
genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens
are either identical or exhibit neutral differences, such as minor variations
in abbreviations or expressions. Leveraging this insight, we introduce **Roads
to Rome (R2R)**, a neural token routing method that selectively utilizes LLMs
only for these critical, path-divergent tokens, while leaving the majority of
token generation to the SLM. We also develop an automatic data generation
pipeline that identifies divergent tokens and generates token-level routing
labels to train the lightweight router. We apply R2R to combine R1-1.5B and
R1-32B models from the DeepSeek family, and evaluate on challenging math,
coding, and QA benchmarks. With an average activated parameter size of 5.6B,
R2R surpasses the average accuracy of R1-7B by 1.6x, outperforming even the
R1-14B model. Compared to R1-32B, it delivers a 2.8x wall-clock speedup with
comparable performance, advancing the Pareto frontier of test-time scaling
efficiency. Our code is available at https://github.com/thu-nics/R2R.
6837bc9c9937bcb69885799c
Tianyu Fu
claimed_verified
2025-05-29T07:42:43.123Z
6445fd9ba56444c355dcbcba
https://cdn-avatars.huggingface.co/v1/production/uploads/6445fd9ba56444c355dcbcba/NCyRCD-MK-yA0_qY6I2y0.png
Tianyu Fu
user
fuvty
6837bc9c9937bcb69885799d
Yi Ge
6837bc9c9937bcb69885799e
Yichen You
claimed_verified
2025-05-29T07:42:40.790Z
66954ebfbcd81f395e9dca37
https://cdn-avatars.huggingface.co/v1/production/uploads/66954ebfbcd81f395e9dca37/0C3m5YdxyXuK7dJBu4AdL.png
Yichen You
user
youyc22
6837bc9c9937bcb69885799f
Enshu Liu
6837bc9c9937bcb6988579a0
Zhihang Yuan
6837bc9c9937bcb6988579a1
Guohao Dai
6837bc9c9937bcb6988579a2
Shengen Yan
6837bc9c9937bcb6988579a3
Huazhong Yang
6837bc9c9937bcb6988579a4
Yu Wang
R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large
  Model Token Routing
Large Language Models (LLMs) achieve impressive reasoning capabilities at the
cost of substantial inference overhead, posing substantial deployment
challenges. Although distilled Small Language Models (SLMs) significantly
enhance efficiency, their performance suffers as they fail to follow LLMs'
reasoning paths. Luckily, we reveal that only a small fraction of tokens
genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens
are either identical or exhibit neutral differences, such as minor variations
in abbreviations or expressions. Leveraging this insight, we introduce **Roads
to Rome (R2R)**, a neural token routing method that selectively utilizes LLMs
only for these critical, path-divergent tokens, while leaving the majority of
token generation to the SLM. We also develop an automatic data generation
pipeline that identifies divergent tokens and generates token-level routing
labels to train the lightweight router. We apply R2R to combine R1-1.5B and
R1-32B models from the DeepSeek family, and evaluate on challenging math,
coding, and QA benchmarks. With an average activated parameter size of 5.6B,
R2R surpasses the average accuracy of R1-7B by 1.6x, outperforming even the
R1-14B model. Compared to R1-32B, it delivers a 2.8x wall-clock speedup with
comparable performance, advancing the Pareto frontier of test-time scaling
efficiency. Our code is available at https://github.com/thu-nics/R2R.
Roads to Rome (R2R) selectively utilizes large language models for critical reasoning tasks to enhance efficiency and performance in lightweight models.
Large Language Models (LLMs) Small Language Models (SLMs) token routing neural token routing token divergence token generation automatic data generation pipeline token-level routing labels R1-1.5B R1-32B math benchmarks coding benchmarks QA benchmarks parameter size activated parameters test-time scaling efficiency pareto frontier