Uncertainty quantification in fine-tuned LLMs using LoRA ensembles
<|reference_start|>Uncertainty quantification in fine-tuned LLMs using LoRA ensembles: Fine-tuning large language models can improve task specific performance, although a general understanding of what the fine-tuned model has learned, forgotten and how to trust its predictions is still missing. We derive principled uncertainty quantification for fine-tuned LLMs with posterior approximations using computationally efficient low-rank adaptation ensembles. We analyze three common multiple-choice datasets using low-rank adaptation ensembles based on Mistral-7b, and draw quantitative and qualitative conclusions on their perceived complexity and model efficacy on the different target domains during and after fine-tuning. In particular, backed by the numerical experiments, we hypothesise about signals from entropic uncertainty measures for data domains that are inherently difficult for a given architecture to learn.<|reference_end|>
@article{balabanov2024uncertainty,
    title={Uncertainty quantification in fine-tuned LLMs using LoRA ensembles},
    author={Oleksandr Balabanov, Hampus Linander},
    journal={arXiv preprint arXiv:2402.12264},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12264},
    primaryClass={cs.LG cs.AI cs.CL stat.ML}
}