title: Preference Leakage: A Contamination Problem in LLM-as-a-judge
summary: Large Language Models (LLMs) as judges and LLM-based data synthesis have
  emerged as two fundamental LLM-driven data annotation methods in model
  development. While their combination significantly enhances the efficiency of
  model training and evaluation, little attention has been given to the potential
  contamination brought by this new model development paradigm. In this work, we
  expose preference leakage, a contamination problem in LLM-as-a-judge caused by
  the relatedness between the synthetic data generators and LLM-based evaluators.
  To study this issue, we first define three common relatednesses between data
  generator LLM and judge LLM: being the same model, having an inheritance
  relationship, and belonging to the same model family. Through extensive
  experiments, we empirically confirm the bias of judges towards their related
  student models caused by preference leakage across multiple LLM baselines and
  benchmarks. Further analysis suggests that preference leakage is a pervasive
  issue that is harder to detect compared to previously identified biases in
  LLM-as-a-judge scenarios. All of these findings imply that preference leakage
  is a widespread and challenging problem in the area of LLM-as-a-judge. We
  release all codes and data at:
  https://github.com/David-Li0406/Preference-Leakage.
paper_authors._id: 67a1ad77d797fac51fa80770
paper_authors.name: Dawei Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:34:59.780Z
paper_authors._id: 6474e1afb68461d5cf7c41cc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6474e1afb68461d5cf7c41cc/bcoiD_qPrjHUBlB259djg.png
paper_authors.fullname: Dawei Li
paper_authors.type: user
paper_authors.user: wjldw
paper_authors._id: 67a1ad77d797fac51fa80771
paper_authors.name: Renliang Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-04T09:39:11.035Z
paper_authors._id: 653a195b0da86d726c9c580c
paper_authors.avatarUrl: /avatars/61649e1d600fdc1edc50ead0dfa99fdd.svg
paper_authors.fullname: Renliang Sun
paper_authors.type: user
paper_authors.user: RLSNLP
paper_authors._id: 67a1ad77d797fac51fa80772
paper_authors.name: Yue Huang
paper_authors._id: 67a1ad77d797fac51fa80773
paper_authors.name: Ming Zhong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T14:54:39.212Z
paper_authors._id: 61d53df2062444ea769d3b79
paper_authors.avatarUrl: /avatars/fa771202368b6b2626a8fdf1c4369239.svg
paper_authors.fullname: Ming Zhong
paper_authors.type: user
paper_authors.user: MingZhong
paper_authors._id: 67a1ad77d797fac51fa80774
paper_authors.name: Bohan Jiang
paper_authors._id: 67a1ad77d797fac51fa80775
paper_authors.name: Jiawei Han
paper_authors._id: 67a1ad77d797fac51fa80776
paper_authors.name: Xiangliang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T17:14:47.621Z
paper_authors._id: 605a97d9b54d35bc67a4ff12
paper_authors.avatarUrl: /avatars/7a48a2dac4e6ebb9e775022e15ddc5a7.svg
paper_authors.fullname: zhangxiangliang
paper_authors.type: user
paper_authors.user: ZhangXiangliang
paper_authors._id: 67a1ad77d797fac51fa80777
paper_authors.name: Wei Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T17:14:54.615Z
paper_authors._id: 62fa0ffe0697d224219a0cb7
paper_authors.avatarUrl: /avatars/f0ef59e1c0cf4ab4fe5cee08d488bd03.svg
paper_authors.fullname: Wei Wang
paper_authors.type: user
paper_authors.user: WeiWang
paper_authors._id: 67a1ad77d797fac51fa80778
paper_authors.name: Huan Liu
paper_title: Preference Leakage: A Contamination Problem in LLM-as-a-judge
paper_summary: Large Language Models (LLMs) as judges and LLM-based data synthesis have
  emerged as two fundamental LLM-driven data annotation methods in model
  development. While their combination significantly enhances the efficiency of
  model training and evaluation, little attention has been given to the potential
  contamination brought by this new model development paradigm. In this work, we
  expose preference leakage, a contamination problem in LLM-as-a-judge caused by
  the relatedness between the synthetic data generators and LLM-based evaluators.
  To study this issue, we first define three common relatednesses between data
  generator LLM and judge LLM: being the same model, having an inheritance
  relationship, and belonging to the same model family. Through extensive
  experiments, we empirically confirm the bias of judges towards their related
  student models caused by preference leakage across multiple LLM baselines and
  benchmarks. Further analysis suggests that preference leakage is a pervasive
  issue that is harder to detect compared to previously identified biases in
  LLM-as-a-judge scenarios. All of these findings imply that preference leakage
  is a widespread and challenging problem in the area of LLM-as-a-judge. We
  release all codes and data at:
  https://github.com/David-Li0406/Preference-Leakage.
paper_ai_summary: Preference leakage, a bias in LLM-as-a-judge systems caused by the relatedness between synthetic data generators and evaluators, has been empirically confirmed and found to be pervasive and challenging.
paper_ai_keywords: LLMs LLM-as-a-judge LLM-based data synthesis data annotation preference leakage relatedness bias benchmarks