title: Judging the Judges: Evaluating Alignment and Vulnerabilities in
    LLMs-as-Judges
summary: Offering a promising solution to the scalability challenges associated with
  human evaluation, the LLM-as-a-judge paradigm is rapidly gaining traction as an
  approach to evaluating large language models (LLMs). However, there are still
  many open questions about the strengths and weaknesses of this paradigm, and
  what potential biases it may hold. In this paper, we present a comprehensive
  study of the performance of various LLMs acting as judges. We leverage TriviaQA
  as a benchmark for assessing objective knowledge reasoning of LLMs and evaluate
  them alongside human annotations which we found to have a high inter-annotator
  agreement. Our study includes 9 judge models and 9 exam taker models -- both
  base and instruction-tuned. We assess the judge model's alignment across
  different model sizes, families, and judge prompts. Among other results, our
  research rediscovers the importance of using Cohen's kappa as a metric of
  alignment as opposed to simple percent agreement, showing that judges with high
  percent agreement can still assign vastly different scores. We find that both
  Llama-3 70B and GPT-4 Turbo have an excellent alignment with humans, but in
  terms of ranking exam taker models, they are outperformed by both JudgeLM-7B
  and the lexical judge Contains, which have up to 34 points lower human
  alignment. Through error analysis and various other studies, including the
  effects of instruction length and leniency bias, we hope to provide valuable
  lessons for using LLMs as judges in the future.
paper_authors._id: 667801ee0d65792ffd18c41a
paper_authors.name: Aman Singh Thakur
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-23T13:47:25.956Z
paper_authors._id: 63b491f3103617b0a5af6b4b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63b491f3103617b0a5af6b4b/4RqcGDxdF8Ny9cqyVuwYg.jpeg
paper_authors.fullname: Aman Singh Thakur
paper_authors.type: user
paper_authors.user: singh96aman
paper_authors._id: 667801ee0d65792ffd18c41b
paper_authors.name: Kartik Choudhary
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-23T22:25:30.094Z
paper_authors._id: 646020c31ba619a54119f8c9
paper_authors.avatarUrl: /avatars/43c7bbac7569e3d63017831f71ecd762.svg
paper_authors.fullname: Kartik Choudhary
paper_authors.type: user
paper_authors.user: kartik727
paper_authors._id: 667801ee0d65792ffd18c41c
paper_authors.name: Venkat Srinik Ramayapally
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-06-23T11:07:28.157Z
paper_authors._id: 65cd743afd5f628b15e6779c
paper_authors.avatarUrl: /avatars/5aa58d7cc9da903399210b9f1f54dd68.svg
paper_authors.fullname: Venkat Srinik Ramayapally
paper_authors.type: user
paper_authors.user: Srinik-1
paper_authors._id: 667801ee0d65792ffd18c41d
paper_authors.name: Sankaran Vaidyanathan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T13:27:35.239Z
paper_authors._id: 6632dea587ce9a8935eb8751
paper_authors.avatarUrl: /avatars/979a72589441400a9dc25b1008de7def.svg
paper_authors.fullname: Sankaran Vaidyanathan
paper_authors.type: user
paper_authors.user: sankaranv
paper_authors._id: 667801ee0d65792ffd18c41e
paper_authors.name: Dieuwke Hupkes
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T13:27:40.612Z
paper_authors._id: 62062bbbc72e929b014f0e49
paper_authors.avatarUrl: /avatars/78c49f863eda2eac82b18261e794fb0f.svg
paper_authors.fullname: Dieuwke Hupkes
paper_authors.type: user
paper_authors.user: dieuwkehupkes
paper_title: Judging the Judges: Evaluating Alignment and Vulnerabilities in
    LLMs-as-Judges
paper_summary: Offering a promising solution to the scalability challenges associated with
  human evaluation, the LLM-as-a-judge paradigm is rapidly gaining traction as an
  approach to evaluating large language models (LLMs). However, there are still
  many open questions about the strengths and weaknesses of this paradigm, and
  what potential biases it may hold. In this paper, we present a comprehensive
  study of the performance of various LLMs acting as judges. We leverage TriviaQA
  as a benchmark for assessing objective knowledge reasoning of LLMs and evaluate
  them alongside human annotations which we found to have a high inter-annotator
  agreement. Our study includes 9 judge models and 9 exam taker models -- both
  base and instruction-tuned. We assess the judge model's alignment across
  different model sizes, families, and judge prompts. Among other results, our
  research rediscovers the importance of using Cohen's kappa as a metric of
  alignment as opposed to simple percent agreement, showing that judges with high
  percent agreement can still assign vastly different scores. We find that both
  Llama-3 70B and GPT-4 Turbo have an excellent alignment with humans, but in
  terms of ranking exam taker models, they are outperformed by both JudgeLM-7B
  and the lexical judge Contains, which have up to 34 points lower human
  alignment. Through error analysis and various other studies, including the
  effects of instruction length and leniency bias, we hope to provide valuable
  lessons for using LLMs as judges in the future.
paper_ai_summary: LLM-as-a-judge method for evaluating LLMs shows promise but varies in performance and alignment with human judges, with smaller models like JudgeLM-7B sometimes outperforming larger ones in ranking tasks.
paper_ai_keywords: LLM-as-a-judge large language models (LLMs) TriviaQA Cohen's kappa alignment instruction-tuned JudgeLM lexical judge leniency bias