Large Language Models, Artificial Intelligence, Data Science, Machine Learning, Editors Pick.
push your own version of a GPTQ 4-bit quantized model on the Hugging Face Hub. As mentioned in the introduction, GPTQ is not the only 4-bit quantization algorithm: GGML and NF4 are excellent alternatives with slightly different scopes. I encourage you to learn more about them and give them a shot!