Machine Learning, Transformers, Data Science, Python, Hugging Face.
the community. quant_model.push_to_hub("edumunozsala/llama-2-7b-int4-GPTQ-python-code-20k") tokenizer.push_to_hub("edumunozsala/llama-2-7b-int4-GPTQ-python-code-20k") In my experiment using GPTQ, the reduction in model size is striking. My fine-tuned Llama 2 7B model with 4-bit weighted 13.5 GB on