EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
Large language models (LLMs) have proven to be very superior to conventional
methods in various tasks. However, their expensive computations and high memory
requirements are prohibitive for deployment. Model quantization is an effective
method for reducing this overhead. The problem is that in most previous works,
the quantized model was calibrated using few samples from the training data,
which might affect the generalization of the quantized LLMs to unknown cases
and tasks. Hence in this work, we explore an important question: Can we design
a data-independent quantization method for LLMs to guarantee its generalization
performance? In this work, we propose EasyQuant, a training-free and
data-independent weight-only quantization algorithm for LLMs. Our observation
indicates that two factors: outliers in the weight and quantization ranges, are
essential for reducing the quantization error. Therefore, in EasyQuant, we
leave the outliers (less than 1%) unchanged and optimize the quantization range
to reduce the reconstruction error. With these methods, we surprisingly find
that EasyQuant achieves comparable performance to the original model. Since
EasyQuant does not depend on any training data, the generalization performance
of quantized LLMs is safely guaranteed. Moreover, EasyQuant can be implemented
in parallel so that the quantized model could be attained in a few minutes even
for LLMs over 100B. To our best knowledge, we are the first work that achieves
almost lossless quantization performance for LLMs under a data-independent
setting and our algorithm runs over 10 times faster than the data-dependent
methods.
65e7da581649b1896ff44ca2
Hanlin Tang
65e7da581649b1896ff44ca3
Yifu Sun
admin_assigned
2024-03-06T10:03:44.524Z
650278893c7d1b2e5bb059b4
/avatars/b0ac7213b4610a01dfef4cbfedd29740.svg
Yifu Sun
user
tomsunyifu
65e7da581649b1896ff44ca4
Decheng Wu
65e7da581649b1896ff44ca5
Kai Liu
65e7da581649b1896ff44ca6
Jianchen Zhu
65e7da581649b1896ff44ca7
Zhanhui Kang
EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
Large language models (LLMs) have proven to be very superior to conventional
methods in various tasks. However, their expensive computations and high memory
requirements are prohibitive for deployment. Model quantization is an effective
method for reducing this overhead. The problem is that in most previous works,
the quantized model was calibrated using few samples from the training data,
which might affect the generalization of the quantized LLMs to unknown cases
and tasks. Hence in this work, we explore an important question: Can we design
a data-independent quantization method for LLMs to guarantee its generalization
performance? In this work, we propose EasyQuant, a training-free and
data-independent weight-only quantization algorithm for LLMs. Our observation
indicates that two factors: outliers in the weight and quantization ranges, are
essential for reducing the quantization error. Therefore, in EasyQuant, we
leave the outliers (less than 1%) unchanged and optimize the quantization range
to reduce the reconstruction error. With these methods, we surprisingly find
that EasyQuant achieves comparable performance to the original model. Since
EasyQuant does not depend on any training data, the generalization performance
of quantized LLMs is safely guaranteed. Moreover, EasyQuant can be implemented
in parallel so that the quantized model could be attained in a few minutes even
for LLMs over 100B. To our best knowledge, we are the first work that achieves
almost lossless quantization performance for LLMs under a data-independent
setting and our algorithm runs over 10 times faster than the data-dependent
methods.
EasyQuant is a data-independent, training-free quantization method for large language models that maintains performance while reducing computational and memory requirements.
large language models model quantization data-independent quantization EasyQuant weight-only quantization outliers quantization range reconstruction error