title: Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
summary: Automatic LLM benchmarks, such as AlpacaEval 2.0, Arena-Hard-Auto, and
  MT-Bench, have become popular for evaluating language models due to their
  cost-effectiveness and scalability compared to human evaluation. Achieving high
  win rates on these benchmarks can significantly boost the promotional impact of
  newly released language models. This promotional benefit may motivate tricks,
  such as manipulating model output length or style to game win rates, even
  though several mechanisms have been developed to control length and disentangle
  style to reduce gameability. Nonetheless, we show that even a "null model" that
  always outputs a constant response (irrelevant to input instructions) can cheat
  automatic benchmarks and achieve top-ranked win rates: an 86.5% LC win rate on
  AlpacaEval 2.0; an 83.0 score on Arena-Hard-Auto; and a 9.55 score on MT-Bench.
  Moreover, the crafted cheating outputs are transferable because we assume that
  the instructions of these benchmarks (e.g., 805 samples of AlpacaEval 2.0) are
  private and cannot be accessed. While our experiments are primarily
  proof-of-concept, an adversary could use LLMs to generate more imperceptible
  cheating responses, unethically benefiting from high win rates and promotional
  impact. Our findings call for the development of anti-cheating mechanisms for
  reliable automatic benchmarks. The code is available at
  https://github.com/sail-sg/Cheating-LLM-Benchmarks.
paper_authors._id: 670747babbfdd3d3073d1e8e
paper_authors.name: Xiaosen Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T08:00:47.197Z
paper_authors._id: 6274a2315d12b3a734adebc9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6274a2315d12b3a734adebc9/zLQDbszAvWh0F2BjdKock.jpeg
paper_authors.fullname: Xiaosen Zheng
paper_authors.type: user
paper_authors.user: xszheng2020
paper_authors._id: 670747babbfdd3d3073d1e8f
paper_authors.name: Tianyu Pang
paper_authors._id: 670747babbfdd3d3073d1e90
paper_authors.name: Chao Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:43:35.992Z
paper_authors._id: 632407c892e07e3ca20aca28
paper_authors.avatarUrl: /avatars/23b51b37b12b51a0947f687d1de4d3b5.svg
paper_authors.fullname: Chao Du
paper_authors.type: user
paper_authors.user: duchao
paper_authors._id: 670747babbfdd3d3073d1e91
paper_authors.name: Qian Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T15:47:23.767Z
paper_authors._id: 612ee6a7b960e78c6d2319d4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/612ee6a7b960e78c6d2319d4/2Hu9BaAyXbyh1vt0v1Qui.jpeg
paper_authors.fullname: Qian Liu
paper_authors.type: user
paper_authors.user: SivilTaram
paper_authors._id: 670747babbfdd3d3073d1e92
paper_authors.name: Jing Jiang
paper_authors._id: 670747babbfdd3d3073d1e93
paper_authors.name: Min Lin
paper_title: Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
paper_summary: Automatic LLM benchmarks, such as AlpacaEval 2.0, Arena-Hard-Auto, and
  MT-Bench, have become popular for evaluating language models due to their
  cost-effectiveness and scalability compared to human evaluation. Achieving high
  win rates on these benchmarks can significantly boost the promotional impact of
  newly released language models. This promotional benefit may motivate tricks,
  such as manipulating model output length or style to game win rates, even
  though several mechanisms have been developed to control length and disentangle
  style to reduce gameability. Nonetheless, we show that even a "null model" that
  always outputs a constant response (irrelevant to input instructions) can cheat
  automatic benchmarks and achieve top-ranked win rates: an 86.5% LC win rate on
  AlpacaEval 2.0; an 83.0 score on Arena-Hard-Auto; and a 9.55 score on MT-Bench.
  Moreover, the crafted cheating outputs are transferable because we assume that
  the instructions of these benchmarks (e.g., 805 samples of AlpacaEval 2.0) are
  private and cannot be accessed. While our experiments are primarily
  proof-of-concept, an adversary could use LLMs to generate more imperceptible
  cheating responses, unethically benefiting from high win rates and promotional
  impact. Our findings call for the development of anti-cheating mechanisms for
  reliable automatic benchmarks. The code is available at
  https://github.com/sail-sg/Cheating-LLM-Benchmarks.
paper_ai_keywords: Automatic LLM benchmarks AlpacaEval 2.0 Arena-Hard-Auto MT-Bench Language models High win rates Promotional impact Cost-effectiveness Scalability Human evaluation Tricks Model output length Disentangle style Gamesability Null model Constant response Quality6.5% LC win rate Transferable Instructions Adversary Imperceptible cheating responses Anti-cheating mechanisms