title: Detecting Pretraining Data from Large Language Models
summary: Although large language models (LLMs) are widely deployed, the data used to
  train them is rarely disclosed. Given the incredible scale of this data, up to
  trillions of tokens, it is all but certain that it includes potentially
  problematic text such as copyrighted materials, personally identifiable
  information, and test data for widely reported reference benchmarks. However,
  we currently have no way to know which data of these types is included or in
  what proportions. In this paper, we study the pretraining data detection
  problem: given a piece of text and black-box access to an LLM without knowing
  the pretraining data, can we determine if the model was trained on the provided
  text? To facilitate this study, we introduce a dynamic benchmark WIKIMIA that
  uses data created before and after model training to support gold truth
  detection. We also introduce a new detection method Min-K% Prob based on a
  simple hypothesis: an unseen example is likely to contain a few outlier words
  with low probabilities under the LLM, while a seen example is less likely to
  have words with such low probabilities. Min-K% Prob can be applied without any
  knowledge about the pretraining corpus or any additional training, departing
  from previous detection methods that require training a reference model on data
  that is similar to the pretraining data. Moreover, our experiments demonstrate
  that Min-K% Prob achieves a 7.4% improvement on WIKIMIA over these previous
  methods. We apply Min-K% Prob to two real-world scenarios, copyrighted book
  detection, and contaminated downstream example detection, and find it a
  consistently effective solution.
paper_authors._id: 6539de890d111541df1f4145
paper_authors.name: Weijia Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:35:07.794Z
paper_authors._id: 6400f2ed568dbe30c9161e47
paper_authors.avatarUrl: /avatars/c55938df5bce82b5d96e592a1ec36a8b.svg
paper_authors.fullname: Weijia Shi
paper_authors.type: user
paper_authors.user: swj0419
paper_authors._id: 6539de890d111541df1f4146
paper_authors.name: Anirudh Ajith
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:35:16.200Z
paper_authors._id: 6244346e799d7b023c675158
paper_authors.avatarUrl: /avatars/e39ca0b6295b229bcef990d2cbe3a295.svg
paper_authors.fullname: Anirudh Ajith
paper_authors.type: user
paper_authors.user: anirudhajith
paper_authors._id: 6539de890d111541df1f4147
paper_authors.name: Mengzhou Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:35:23.435Z
paper_authors._id: 6241126dedf27fda71c6b573
paper_authors.avatarUrl: /avatars/ac106139c0e5967776f39e2755ca0792.svg
paper_authors.fullname: Mengzhou Xia
paper_authors.type: user
paper_authors.user: mengzhouxia
paper_authors._id: 6539de890d111541df1f4148
paper_authors.name: Yangsibo Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:35:48.652Z
paper_authors._id: 5f32f7f77e58354338621556
paper_authors.avatarUrl: /avatars/b2d702848d7b6c404c28b9959f050fdc.svg
paper_authors.fullname: Yangsibo Huang
paper_authors.type: user
paper_authors.user: hazelsuko
paper_authors._id: 6539de890d111541df1f4149
paper_authors.name: Daogao Liu
paper_authors._id: 6539de890d111541df1f414a
paper_authors.name: Terra Blevins
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:36:10.488Z
paper_authors._id: 64528b183f80ad88c7791862
paper_authors.avatarUrl: /avatars/f25635c484d86b23db42289fc29eb73a.svg
paper_authors.fullname: Terra Blevins
paper_authors.type: user
paper_authors.user: blvns
paper_authors._id: 6539de890d111541df1f414b
paper_authors.name: Danqi Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-26T11:36:51.728Z
paper_authors._id: 653a4ca8669b77a9f8034ec0
paper_authors.avatarUrl: /avatars/005ba9ead6b95b9ee8a31d050de77cee.svg
paper_authors.fullname: Danqi Chen
paper_authors.type: user
paper_authors.user: cdq10131
paper_authors._id: 6539de890d111541df1f414c
paper_authors.name: Luke Zettlemoyer
paper_title: Detecting Pretraining Data from Large Language Models
paper_summary: Although large language models (LLMs) are widely deployed, the data used to
  train them is rarely disclosed. Given the incredible scale of this data, up to
  trillions of tokens, it is all but certain that it includes potentially
  problematic text such as copyrighted materials, personally identifiable
  information, and test data for widely reported reference benchmarks. However,
  we currently have no way to know which data of these types is included or in
  what proportions. In this paper, we study the pretraining data detection
  problem: given a piece of text and black-box access to an LLM without knowing
  the pretraining data, can we determine if the model was trained on the provided
  text? To facilitate this study, we introduce a dynamic benchmark WIKIMIA that
  uses data created before and after model training to support gold truth
  detection. We also introduce a new detection method Min-K% Prob based on a
  simple hypothesis: an unseen example is likely to contain a few outlier words
  with low probabilities under the LLM, while a seen example is less likely to
  have words with such low probabilities. Min-K% Prob can be applied without any
  knowledge about the pretraining corpus or any additional training, departing
  from previous detection methods that require training a reference model on data
  that is similar to the pretraining data. Moreover, our experiments demonstrate
  that Min-K% Prob achieves a 7.4% improvement on WIKIMIA over these previous
  methods. We apply Min-K% Prob to two real-world scenarios, copyrighted book
  detection, and contaminated downstream example detection, and find it a
  consistently effective solution.
paper_ai_summary: Researchers propose Min-K% Prob, a method for detecting pretraining data in large language models without needing knowledge of the pretraining corpus, using a benchmark that supports gold truth detection.
paper_ai_keywords: large language models pretraining data detection black-box access WIKIMIA Min-K% Prob outlier words low probabilities