Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning
  Models
Recent advancements in reasoning language models have demonstrated remarkable
performance in complex tasks, but their extended chain-of-thought reasoning
process increases inference overhead. While quantization has been widely
adopted to reduce the inference cost of large language models, its impact on
reasoning models remains understudied. In this study, we conduct the first
systematic study on quantized reasoning models, evaluating the open-sourced
DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B
parameters, and QwQ-32B. Our investigation covers weight, KV cache, and
activation quantization using state-of-the-art algorithms at varying
bit-widths, with extensive evaluation across mathematical (AIME, MATH-500),
scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our
findings reveal that while lossless quantization can be achieved with W8A8 or
W4A16 quantization, lower bit-widths introduce significant accuracy risks. We
further identify model size, model origin, and task difficulty as critical
determinants of performance. Contrary to expectations, quantized models do not
exhibit increased output lengths. In addition, strategically scaling the model
sizes or reasoning steps can effectively enhance the performance. All quantized
models and codes will be open-sourced in
https://github.com/ruikangliu/Quantized-Reasoning-Models.
67f4a8ead83d88e30c450332
Ruikang Liu
claimed_verified
2025-04-08T06:51:24.004Z
6411c22dd52c57f628f7c331
https://cdn-avatars.huggingface.co/v1/production/uploads/1678885386428-noauth.jpeg
ruikang liu
user
ruikangliu
67f4a8ead83d88e30c450333
Yuxuan Sun
67f4a8ead83d88e30c450334
Manyi Zhang
admin_assigned
2025-04-08T10:31:35.125Z
66baf914324dd1598b7a7ce5
/avatars/9b54474a053589b4bfa704d0305cbca7.svg
manyi zhang
user
manyi2024
67f4a8ead83d88e30c450335
Haoli Bai
admin_assigned
2025-04-08T10:31:26.542Z
667e1a0fa0ffad3dbc60df9d
/avatars/e16c7a3620ae5c82b7b636296a975ae1.svg
Haoli Bai
user
baihaoli
67f4a8ead83d88e30c450336
Xianzhi Yu
67f4a8ead83d88e30c450337
Tiezheng Yu
admin_assigned
2025-04-08T10:31:10.674Z
61bd88aa94bd578f7286181a
/avatars/0f40dfd7accc558f9201103e3e4ec007.svg
Tiezheng Yu
user
Tiezheng
67f4a8ead83d88e30c450338
Chun Yuan
67f4a8ead83d88e30c450339
Lu Hou
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning
  Models
Recent advancements in reasoning language models have demonstrated remarkable
performance in complex tasks, but their extended chain-of-thought reasoning
process increases inference overhead. While quantization has been widely
adopted to reduce the inference cost of large language models, its impact on
reasoning models remains understudied. In this study, we conduct the first
systematic study on quantized reasoning models, evaluating the open-sourced
DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B
parameters, and QwQ-32B. Our investigation covers weight, KV cache, and
activation quantization using state-of-the-art algorithms at varying
bit-widths, with extensive evaluation across mathematical (AIME, MATH-500),
scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our
findings reveal that while lossless quantization can be achieved with W8A8 or
W4A16 quantization, lower bit-widths introduce significant accuracy risks. We
further identify model size, model origin, and task difficulty as critical
determinants of performance. Contrary to expectations, quantized models do not
exhibit increased output lengths. In addition, strategically scaling the model
sizes or reasoning steps can effectively enhance the performance. All quantized
models and codes will be open-sourced in
https://github.com/ruikangliu/Quantized-Reasoning-Models.
A systematic study on quantized reasoning models evaluates the impact of quantization on performance and identifies critical factors affecting accuracy and inference cost.
quantization reasoning models DeepSeek-R1-Distilled Qwen LLaMA weight quantization KV cache quantization activation quantization AIME MATH-500 GPQA LiveCodeBench W8A8 W4A16 quantized models