Llm, Model Quantization, Deep Learning, Machine Learning, Large Language Models.
memory reduction by carefully designing 4-bit quantization, reducing the average memory requirements for finetuning a 65 billion parameter model from over 780GB of GPU memory to less than 48GB, without degrading runtime or predictive performance compared to a fully finetuned baseline using 16-bit