Calibrating the Confidence of Large Language Models by Eliciting Fidelity
<|reference_start|>Calibrating the Confidence of Large Language Models by Eliciting Fidelity: Large language models optimized with techniques like RLHF have achieved good alignment in being helpful and harmless. However, post-alignment, these language models often exhibit overconfidence, where the expressed confidence does not accurately calibrate with their correctness rate. In this paper, we decompose the language model confidence into the \textit{Uncertainty} about the question and the \textit{Fidelity} to the answer generated by language models. Then, we propose a plug-and-play method to estimate the confidence of language models. Our method has shown good calibration performance by conducting experiments with 6 RLHF-LMs on four MCQA datasets. Moreover, we propose two novel metrics, IPR and CE, to evaluate the calibration of the model, and we have conducted a detailed discussion on \textit{Truly Well-Calibrated Confidence}. Our method could serve as a strong baseline, and we hope that this work will provide some insights into the model confidence calibration.<|reference_end|>
@article{zhang2024calibrating,
    title={Calibrating the Confidence of Large Language Models by Eliciting
  Fidelity},
    author={Mozhi Zhang, Mianqiu Huang, Rundong Shi, Linsen Guo, Chong Peng, Peng
  Yan, Yaqian Zhou, Xipeng Qiu},
    journal={arXiv preprint arXiv:2404.02655},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.02655},
    primaryClass={cs.CL}
}