title: Measuring memorization through probabilistic discoverable extraction
summary: Large language models (LLMs) are susceptible to memorizing training data,
  raising concerns due to the potential extraction of sensitive information.
  Current methods to measure memorization rates of LLMs, primarily discoverable
  extraction (Carlini et al., 2022), rely on single-sequence greedy sampling,
  potentially underestimating the true extent of memorization. This paper
  introduces a probabilistic relaxation of discoverable extraction that
  quantifies the probability of extracting a target sequence within a set of
  generated samples, considering various sampling schemes and multiple attempts.
  This approach addresses the limitations of reporting memorization rates through
  discoverable extraction by accounting for the probabilistic nature of LLMs and
  user interaction patterns. Our experiments demonstrate that this probabilistic
  measure can reveal cases of higher memorization rates compared to rates found
  through discoverable extraction. We further investigate the impact of different
  sampling schemes on extractability, providing a more comprehensive and
  realistic assessment of LLM memorization and its associated risks. Our
  contributions include a new probabilistic memorization definition, empirical
  evidence of its effectiveness, and a thorough evaluation across different
  models, sizes, sampling schemes, and training data repetitions.
paper_authors._id: 67226a32ea6e0084fc0f029c
paper_authors.name: Jamie Hayes
paper_authors._id: 67226a32ea6e0084fc0f029d
paper_authors.name: Marika Swanberg
paper_authors._id: 67226a32ea6e0084fc0f029e
paper_authors.name: Harsh Chaudhari
paper_authors._id: 67226a32ea6e0084fc0f029f
paper_authors.name: Itay Yona
paper_authors._id: 67226a32ea6e0084fc0f02a0
paper_authors.name: Ilia Shumailov
paper_title: Measuring memorization through probabilistic discoverable extraction
paper_summary: Large language models (LLMs) are susceptible to memorizing training data,
  raising concerns due to the potential extraction of sensitive information.
  Current methods to measure memorization rates of LLMs, primarily discoverable
  extraction (Carlini et al., 2022), rely on single-sequence greedy sampling,
  potentially underestimating the true extent of memorization. This paper
  introduces a probabilistic relaxation of discoverable extraction that
  quantifies the probability of extracting a target sequence within a set of
  generated samples, considering various sampling schemes and multiple attempts.
  This approach addresses the limitations of reporting memorization rates through
  discoverable extraction by accounting for the probabilistic nature of LLMs and
  user interaction patterns. Our experiments demonstrate that this probabilistic
  measure can reveal cases of higher memorization rates compared to rates found
  through discoverable extraction. We further investigate the impact of different
  sampling schemes on extractability, providing a more comprehensive and
  realistic assessment of LLM memorization and its associated risks. Our
  contributions include a new probabilistic memorization definition, empirical
  evidence of its effectiveness, and a thorough evaluation across different
  models, sizes, sampling schemes, and training data repetitions.
paper_ai_summary: A probabilistic method for measuring memorization in large language models provides a more comprehensive assessment by considering various sampling schemes and multiple attempts compared to deterministic discoverable extraction.
paper_ai_keywords: large language models memorization discoverable extraction probabilistic relaxation sampling schemes memorization rates user interaction patterns probabilistic memorization definition empirical evidence