On the Limitations of Fine-tuned Judge Models for LLM Evaluation
<|reference_start|>On the Limitations of Fine-tuned Judge Models for LLM Evaluation: Recently, there has been a growing trend of utilizing Large Language Model (LLM) to evaluate the quality of other LLMs. Many studies have employed proprietary close-source models, especially GPT-4, as the evaluator. Alternatively, other works have fine-tuned judge models based on open-source LLMs as the evaluator. While the fine-tuned judge models are claimed to achieve comparable evaluation capability with GPT-4, in this study, we conduct an empirical study of judge models. Our findings indicate that although the fine-tuned judge models achieve high performance on in-domain test sets, even surpassing GPT-4, they underperform GPT-4 across several dimensions, including generalizability, fairness, aspect-specific evaluation, and scalability. We also reveal that the fine-tuned judge model inherently operates as a task-specific classifier, consequently imposing the limitations. Finally, we propose an effective indicator to measure the reliability of fine-tuned judges, with the aim of maximizing their utility in LLM evaluation.<|reference_end|>
@article{huang2024an,
    title={An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned
  Judge Model is not a General Substitute for GPT-4},
    author={Hui Huang, Yingqi Qu, Xingyuan Bu, Hongli Zhou, Jing Liu, Muyun Yang,
  Bing Xu, Tiejun Zhao},
    journal={arXiv preprint arXiv:2403.02839},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.02839},
    primaryClass={cs.CL}
}