title: Challenges in Trustworthy Human Evaluation of Chatbots
summary: Open community-driven platforms like Chatbot Arena that collect user
  preference data from site visitors have gained a reputation as one of the most
  trustworthy publicly available benchmarks for LLM performance. While now
  standard, it is tricky to implement effective guardrails to collect
  high-quality annotations from humans. In this paper, we demonstrate that three
  sources of bad annotations, both malicious and otherwise, can corrupt the
  reliability of open leaderboard rankings. In particular, we show that only 10\%
  of poor quality votes by apathetic (site visitors not appropriately
  incentivized to give correct votes) or adversarial (bad actors seeking to
  inflate the ranking of a target model) annotators can change the rankings of
  models by up to 5 places on the leaderboard. Finally, we discuss open
  challenges in ensuring high-quality human annotations.
paper_authors._id: 67530912232ea247a627e51a
paper_authors.name: Wenting Zhao
paper_authors._id: 67530912232ea247a627e51b
paper_authors.name: Alexander M. Rush
paper_authors._id: 67530912232ea247a627e51c
paper_authors.name: Tanya Goyal
paper_title: Challenges in Trustworthy Human Evaluation of Chatbots
paper_summary: Open community-driven platforms like Chatbot Arena that collect user
  preference data from site visitors have gained a reputation as one of the most
  trustworthy publicly available benchmarks for LLM performance. While now
  standard, it is tricky to implement effective guardrails to collect
  high-quality annotations from humans. In this paper, we demonstrate that three
  sources of bad annotations, both malicious and otherwise, can corrupt the
  reliability of open leaderboard rankings. In particular, we show that only 10\%
  of poor quality votes by apathetic (site visitors not appropriately
  incentivized to give correct votes) or adversarial (bad actors seeking to
  inflate the ranking of a target model) annotators can change the rankings of
  models by up to 5 places on the leaderboard. Finally, we discuss open
  challenges in ensuring high-quality human annotations.
paper_ai_keywords: Chatbot Arena user preference data publicly available benchmarks LLM performance guardrails high-quality annotations malicious bad annotations apathetic annotators adversarial annotators leaderboard rankings poor quality votes