Llm, Model Quantization, Deep Learning, Machine Learning, Large Language Models.
for Model Quantization It mainly includes the following aspects: Weights: Quantizing weights is the most common and popular approach, it can reduce the model size, memory usage, and space occupation. Activations: In practice, activations often account for the majority of memory usage. Therefore,