Machine Learning, Transformers, Data Science, Python, Hugging Face.
description of the model in the hugging Face Hub. In my notebook, we will describe how to perform inference on the examples mentioned. Quantize the model using auto-gptq, 🤗 transformers, and optimum The GPTQ quantization consumes a lot of GPU VRAM, for that reason we need to execute it in an A100