title: DyePack: Provably Flagging Test Set Contamination in LLMs Using
    Backdoors
summary: Open benchmarks are essential for evaluating and advancing large language
  models, offering reproducibility and transparency. However, their accessibility
  makes them likely targets of test set contamination. In this work, we introduce
  DyePack, a framework that leverages backdoor attacks to identify models that
  used benchmark test sets during training, without requiring access to the loss,
  logits, or any internal details of the model. Like how banks mix dye packs with
  their money to mark robbers, DyePack mixes backdoor samples with the test data
  to flag models that trained on it. We propose a principled design incorporating
  multiple backdoors with stochastic targets, enabling exact false positive rate
  (FPR) computation when flagging every model. This provably prevents false
  accusations while providing strong evidence for every detected case of
  contamination. We evaluate DyePack on five models across three datasets,
  covering both multiple-choice and open-ended generation tasks. For
  multiple-choice questions, it successfully detects all contaminated models with
  guaranteed FPRs as low as 0.000073% on MMLU-Pro and 0.000017% on Big-Bench-Hard
  using eight backdoors. For open-ended generation tasks, it generalizes well and
  identifies all contaminated models on Alpaca with a guaranteed false positive
  rate of just 0.127% using six backdoors.
paper_authors._id: 6839f87c49d173e7b23f220b
paper_authors.name: Yize Cheng
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-30T18:27:47.059Z
paper_authors._id: 66fa2c61c25c3fcb32f9f131
paper_authors.avatarUrl: /avatars/05387c30f2d1803fa0a5b176c3706772.svg
paper_authors.fullname: Yize Cheng
paper_authors.type: user
paper_authors.user: yizecheng
paper_authors._id: 6839f87c49d173e7b23f220c
paper_authors.name: Wenxiao Wang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-30T18:28:18.975Z
paper_authors._id: 659dc02d72238596c24d49f5
paper_authors.avatarUrl: /avatars/d4600d23ccc72f296fab7f626d5895e7.svg
paper_authors.fullname: Wenxiao Wang
paper_authors.type: user
paper_authors.user: wangwenxiao
paper_authors._id: 6839f87c49d173e7b23f220d
paper_authors.name: Mazda Moayeri
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-30T20:01:34.913Z
paper_authors._id: 63449874ee1504dbcd59af3d
paper_authors.avatarUrl: /avatars/57a805d82d16de9544c98585bd7a3e55.svg
paper_authors.fullname: MazdaM
paper_authors.type: user
paper_authors.user: mmoayeri
paper_authors._id: 6839f87c49d173e7b23f220e
paper_authors.name: Soheil Feizi
paper_title: DyePack: Provably Flagging Test Set Contamination in LLMs Using
    Backdoors
paper_summary: Open benchmarks are essential for evaluating and advancing large language
  models, offering reproducibility and transparency. However, their accessibility
  makes them likely targets of test set contamination. In this work, we introduce
  DyePack, a framework that leverages backdoor attacks to identify models that
  used benchmark test sets during training, without requiring access to the loss,
  logits, or any internal details of the model. Like how banks mix dye packs with
  their money to mark robbers, DyePack mixes backdoor samples with the test data
  to flag models that trained on it. We propose a principled design incorporating
  multiple backdoors with stochastic targets, enabling exact false positive rate
  (FPR) computation when flagging every model. This provably prevents false
  accusations while providing strong evidence for every detected case of
  contamination. We evaluate DyePack on five models across three datasets,
  covering both multiple-choice and open-ended generation tasks. For
  multiple-choice questions, it successfully detects all contaminated models with
  guaranteed FPRs as low as 0.000073% on MMLU-Pro and 0.000017% on Big-Bench-Hard
  using eight backdoors. For open-ended generation tasks, it generalizes well and
  identifies all contaminated models on Alpaca with a guaranteed false positive
  rate of just 0.127% using six backdoors.
paper_ai_summary: DyePack, a framework using backdoor attacks, identifies models that leveraged benchmark test sets during training by introducing benign backdoor samples, ensuring precise false positive rates while preventing false accusations.
paper_ai_keywords: backdoor attacks test set contamination false positive rate FPR DyePack multiple backdoors stochastic targets MMLU-Pro Big-Bench-Hard Alpaca multiple-choice questions open-ended generation tasks