Large Language Models, Quantization, Machine Learning, Data Science, Hands On Tutorials.
consists of quantizing the weights using zero-point and absmax quantization. In the following example, we apply these techniques to the first attention layer of GPT-2 to see the results. # Extract weights of the first layer weights = model.transformer.h[0].attn.c_attn.weight.data print("Original