Artificial Intelligence, Deep Learning, Neural Networks, Large Language Models, Machine Learning.
Quantization is a frequently used method to reduce the memory and computational capacity requirements of our machine learning models. During quantization, we use less precise data formats, such as 8-bit integers (int8), to represent weights. This not only reduces computational capacity and memory