TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness
<|reference_start|>TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness: Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, prompting a surge in their practical applications. However, concerns have arisen regarding the trustworthiness of LLMs outputs, particularly in closed-book question-answering tasks, where non-experts may struggle to identify inaccuracies due to the absence of contextual or ground truth information. This paper introduces TrustScore, a framework based on the concept of Behavioral Consistency, which evaluates whether an LLMs response aligns with its intrinsic knowledge. Additionally, TrustScore can seamlessly integrate with fact-checking methods, which assesses alignment with external knowledge sources. The experimental results show that TrustScore achieves strong correlations with human judgments, surpassing existing reference-free metrics, and achieving results on par with reference-based metrics.<|reference_end|>
@article{zheng2024trustscore:,
    title={TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness},
    author={Danna Zheng, Danyang Liu, Mirella Lapata, Jeff Z. Pan},
    journal={arXiv preprint arXiv:2402.12545},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12545},
    primaryClass={cs.CL}
}