Self-Evaluation of Large Language Model based on Glass-box Features
<|reference_start|>Self-Evaluation of Large Language Model based on Glass-box Features: The proliferation of open-source Large Language Models (LLMs) underscores the pressing need for evaluation methods. Existing works primarily rely on external evaluators, focusing on training and prompting strategies. However, a crucial aspect, model-aware glass-box features, is overlooked. In this study, we explore the utility of glass-box features under the scenario of self-evaluation, namely applying an LLM to evaluate its own output. We investigate various glass-box feature groups and discovered that the softmax distribution serves as a reliable quality indicator for self-evaluation. Experimental results on public benchmarks validate the feasibility of self-evaluation of LLMs using glass-box features.<|reference_end|>
@article{huang2024self-evaluation,
    title={Self-Evaluation of Large Language Model based on Glass-box Features},
    author={Hui Huang, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, Tiejun Zhao,
  Wenpeng Lu},
    journal={arXiv preprint arXiv:2403.04222},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.04222},
    primaryClass={cs.CL}
}