title: Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning
    Models
summary: Recent advancements in reasoning language models have demonstrated remarkable
  performance in complex tasks, but their extended chain-of-thought reasoning
  process increases inference overhead. While quantization has been widely
  adopted to reduce the inference cost of large language models, its impact on
  reasoning models remains understudied. In this study, we conduct the first
  systematic study on quantized reasoning models, evaluating the open-sourced
  DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B
  parameters, and QwQ-32B. Our investigation covers weight, KV cache, and
  activation quantization using state-of-the-art algorithms at varying
  bit-widths, with extensive evaluation across mathematical (AIME, MATH-500),
  scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our
  findings reveal that while lossless quantization can be achieved with W8A8 or
  W4A16 quantization, lower bit-widths introduce significant accuracy risks. We
  further identify model size, model origin, and task difficulty as critical
  determinants of performance. Contrary to expectations, quantized models do not
  exhibit increased output lengths. In addition, strategically scaling the model
  sizes or reasoning steps can effectively enhance the performance. All quantized
  models and codes will be open-sourced in
  https://github.com/ruikangliu/Quantized-Reasoning-Models.
paper_authors._id: 67f4a8ead83d88e30c450332
paper_authors.name: Ruikang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:51:24.004Z
paper_authors._id: 6411c22dd52c57f628f7c331
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678885386428-noauth.jpeg
paper_authors.fullname: ruikang liu
paper_authors.type: user
paper_authors.user: ruikangliu
paper_authors._id: 67f4a8ead83d88e30c450333
paper_authors.name: Yuxuan Sun
paper_authors._id: 67f4a8ead83d88e30c450334
paper_authors.name: Manyi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:31:35.125Z
paper_authors._id: 66baf914324dd1598b7a7ce5
paper_authors.avatarUrl: /avatars/9b54474a053589b4bfa704d0305cbca7.svg
paper_authors.fullname: manyi zhang
paper_authors.type: user
paper_authors.user: manyi2024
paper_authors._id: 67f4a8ead83d88e30c450335
paper_authors.name: Haoli Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:31:26.542Z
paper_authors._id: 667e1a0fa0ffad3dbc60df9d
paper_authors.avatarUrl: /avatars/e16c7a3620ae5c82b7b636296a975ae1.svg
paper_authors.fullname: Haoli Bai
paper_authors.type: user
paper_authors.user: baihaoli
paper_authors._id: 67f4a8ead83d88e30c450336
paper_authors.name: Xianzhi Yu
paper_authors._id: 67f4a8ead83d88e30c450337
paper_authors.name: Tiezheng Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:31:10.674Z
paper_authors._id: 61bd88aa94bd578f7286181a
paper_authors.avatarUrl: /avatars/0f40dfd7accc558f9201103e3e4ec007.svg
paper_authors.fullname: Tiezheng Yu
paper_authors.type: user
paper_authors.user: Tiezheng
paper_authors._id: 67f4a8ead83d88e30c450338
paper_authors.name: Chun Yuan
paper_authors._id: 67f4a8ead83d88e30c450339
paper_authors.name: Lu Hou
paper_title: Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning
    Models
paper_summary: Recent advancements in reasoning language models have demonstrated remarkable
  performance in complex tasks, but their extended chain-of-thought reasoning
  process increases inference overhead. While quantization has been widely
  adopted to reduce the inference cost of large language models, its impact on
  reasoning models remains understudied. In this study, we conduct the first
  systematic study on quantized reasoning models, evaluating the open-sourced
  DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B
  parameters, and QwQ-32B. Our investigation covers weight, KV cache, and
  activation quantization using state-of-the-art algorithms at varying
  bit-widths, with extensive evaluation across mathematical (AIME, MATH-500),
  scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our
  findings reveal that while lossless quantization can be achieved with W8A8 or
  W4A16 quantization, lower bit-widths introduce significant accuracy risks. We
  further identify model size, model origin, and task difficulty as critical
  determinants of performance. Contrary to expectations, quantized models do not
  exhibit increased output lengths. In addition, strategically scaling the model
  sizes or reasoning steps can effectively enhance the performance. All quantized
  models and codes will be open-sourced in
  https://github.com/ruikangliu/Quantized-Reasoning-Models.
paper_ai_summary: A systematic study on quantized reasoning models evaluates the impact of quantization on performance and identifies critical factors affecting accuracy and inference cost.
paper_ai_keywords: quantization reasoning models DeepSeek-R1-Distilled Qwen LLaMA weight quantization KV cache quantization activation quantization AIME MATH-500 GPQA LiveCodeBench W8A8 W4A16 quantized models