An additional limitation is that we did not evaluate different bit-precisions, such as using 3-bit base
models, or different adapter methods. Besides LoRA, there is also a wide variety Parameter Efficient
FineTuning (PEFT) methods that have been shown to work well. However, it is unclear if these
methods scale to large models. We used LoRA as many results established its robustness but other
adapters might yield better performance. Since finetuning after quantization seems to recover most of
the information that is lost during quantization this might enable much more aggressive quantization.
For example, 3-bit GPTQ quantization of the basemodel with LoRA might also yield 16-bit full
finetuning performance after finetuning.
9 Broader Impacts
Our QLORA finetuning method is the first method that enables the finetuning of 33B parameter
models on a single consumer GPU and 65B parameter models on a single professional GPU, while
QLORA