title: EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
summary: Large language models (LLMs) have proven to be very superior to conventional
  methods in various tasks. However, their expensive computations and high memory
  requirements are prohibitive for deployment. Model quantization is an effective
  method for reducing this overhead. The problem is that in most previous works,
  the quantized model was calibrated using few samples from the training data,
  which might affect the generalization of the quantized LLMs to unknown cases
  and tasks. Hence in this work, we explore an important question: Can we design
  a data-independent quantization method for LLMs to guarantee its generalization
  performance? In this work, we propose EasyQuant, a training-free and
  data-independent weight-only quantization algorithm for LLMs. Our observation
  indicates that two factors: outliers in the weight and quantization ranges, are
  essential for reducing the quantization error. Therefore, in EasyQuant, we
  leave the outliers (less than 1%) unchanged and optimize the quantization range
  to reduce the reconstruction error. With these methods, we surprisingly find
  that EasyQuant achieves comparable performance to the original model. Since
  EasyQuant does not depend on any training data, the generalization performance
  of quantized LLMs is safely guaranteed. Moreover, EasyQuant can be implemented
  in parallel so that the quantized model could be attained in a few minutes even
  for LLMs over 100B. To our best knowledge, we are the first work that achieves
  almost lossless quantization performance for LLMs under a data-independent
  setting and our algorithm runs over 10 times faster than the data-dependent
  methods.
paper_authors._id: 65e7da581649b1896ff44ca2
paper_authors.name: Hanlin Tang
paper_authors._id: 65e7da581649b1896ff44ca3
paper_authors.name: Yifu Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T10:03:44.524Z
paper_authors._id: 650278893c7d1b2e5bb059b4
paper_authors.avatarUrl: /avatars/b0ac7213b4610a01dfef4cbfedd29740.svg
paper_authors.fullname: Yifu Sun
paper_authors.type: user
paper_authors.user: tomsunyifu
paper_authors._id: 65e7da581649b1896ff44ca4
paper_authors.name: Decheng Wu
paper_authors._id: 65e7da581649b1896ff44ca5
paper_authors.name: Kai Liu
paper_authors._id: 65e7da581649b1896ff44ca6
paper_authors.name: Jianchen Zhu
paper_authors._id: 65e7da581649b1896ff44ca7
paper_authors.name: Zhanhui Kang
paper_title: EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
paper_summary: Large language models (LLMs) have proven to be very superior to conventional
  methods in various tasks. However, their expensive computations and high memory
  requirements are prohibitive for deployment. Model quantization is an effective
  method for reducing this overhead. The problem is that in most previous works,
  the quantized model was calibrated using few samples from the training data,
  which might affect the generalization of the quantized LLMs to unknown cases
  and tasks. Hence in this work, we explore an important question: Can we design
  a data-independent quantization method for LLMs to guarantee its generalization
  performance? In this work, we propose EasyQuant, a training-free and
  data-independent weight-only quantization algorithm for LLMs. Our observation
  indicates that two factors: outliers in the weight and quantization ranges, are
  essential for reducing the quantization error. Therefore, in EasyQuant, we
  leave the outliers (less than 1%) unchanged and optimize the quantization range
  to reduce the reconstruction error. With these methods, we surprisingly find
  that EasyQuant achieves comparable performance to the original model. Since
  EasyQuant does not depend on any training data, the generalization performance
  of quantized LLMs is safely guaranteed. Moreover, EasyQuant can be implemented
  in parallel so that the quantized model could be attained in a few minutes even
  for LLMs over 100B. To our best knowledge, we are the first work that achieves
  almost lossless quantization performance for LLMs under a data-independent
  setting and our algorithm runs over 10 times faster than the data-dependent
  methods.
paper_ai_summary: EasyQuant is a data-independent, training-free quantization method for large language models that maintains performance while reducing computational and memory requirements.
paper_ai_keywords: large language models model quantization data-independent quantization EasyQuant weight-only quantization outliers quantization range reconstruction error