Llm, Model Quantization, Deep Learning, Machine Learning, Large Language Models.
Quantization of deep learning models is a memory optimization technique that reduces memory space by sacrificing some accuracy. In the era of large language models, quantization is an essential technique during the training, finetuning and inference stages. For example, qlora achieves significant