title: Controlling the Extraction of Memorized Data from Large Language Models
    via Prompt-Tuning
summary: Large Language Models (LLMs) are known to memorize significant portions of
  their training data. Parts of this memorized content have been shown to be
  extractable by simply querying the model, which poses a privacy risk. We
  present a novel approach which uses prompt-tuning to control the extraction
  rates of memorized content in LLMs. We present two prompt training strategies
  to increase and decrease extraction rates, which correspond to an attack and a
  defense, respectively. We demonstrate the effectiveness of our techniques by
  using models from the GPT-Neo family on a public benchmark. For the 1.3B
  parameter GPT-Neo model, our attack yields a 9.3 percentage point increase in
  extraction rate compared to our baseline. Our defense can be tuned to achieve
  different privacy-utility trade-offs by a user-specified hyperparameter. We
  achieve an extraction rate reduction of up to 97.7% relative to our baseline,
  with a perplexity increase of 16.9%.
paper_authors._id: 646ae91531fbfc5df82027a3
paper_authors.name: Mustafa Safa Ozdayi
paper_authors._id: 646ae91531fbfc5df82027a4
paper_authors.name: Charith Peris
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-22T12:41:16.719Z
paper_authors._id: 626029fdd47e3dbae32d89e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1651094348689-626029fdd47e3dbae32d89e2.jpeg
paper_authors.fullname: Charith Peris
paper_authors.type: user
paper_authors.user: cperiz
paper_authors._id: 646ae91531fbfc5df82027a5
paper_authors.name: Jack FitzGerald
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-22T12:41:58.568Z
paper_authors._id: 6269d5cda6a7bba9e46dbea4
paper_authors.avatarUrl: /avatars/6e2c4751ae8bad23fbb5af61ab535ee8.svg
paper_authors.fullname: Jack FitzGerald
paper_authors.type: user
paper_authors.user: jgmf
paper_authors._id: 646ae91531fbfc5df82027a6
paper_authors.name: Christophe Dupuy
paper_authors._id: 646ae91531fbfc5df82027a7
paper_authors.name: Jimit Majmudar
paper_authors._id: 646ae91531fbfc5df82027a8
paper_authors.name: Haidar Khan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:38:23.601Z
paper_authors._id: 6556a6f90bd9dbcb06a257c4
paper_authors.avatarUrl: /avatars/b315ba3a8e676ae7719e73a378cc0e07.svg
paper_authors.fullname: Haidar Khan
paper_authors.type: user
paper_authors.user: haidark1
paper_authors._id: 646ae91531fbfc5df82027a9
paper_authors.name: Rahil Parikh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-22T12:42:55.293Z
paper_authors._id: 645ae69ce7e8a4b57bdc08f8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645ae69ce7e8a4b57bdc08f8/g26MrrfmzS8GxCuoccOsV.jpeg
paper_authors.fullname: Rahil Parikh
paper_authors.type: user
paper_authors.user: rhlprkh
paper_authors._id: 646ae91531fbfc5df82027aa
paper_authors.name: Rahul Gupta
paper_title: Controlling the Extraction of Memorized Data from Large Language Models
    via Prompt-Tuning
paper_summary: Large Language Models (LLMs) are known to memorize significant portions of
  their training data. Parts of this memorized content have been shown to be
  extractable by simply querying the model, which poses a privacy risk. We
  present a novel approach which uses prompt-tuning to control the extraction
  rates of memorized content in LLMs. We present two prompt training strategies
  to increase and decrease extraction rates, which correspond to an attack and a
  defense, respectively. We demonstrate the effectiveness of our techniques by
  using models from the GPT-Neo family on a public benchmark. For the 1.3B
  parameter GPT-Neo model, our attack yields a 9.3 percentage point increase in
  extraction rate compared to our baseline. Our defense can be tuned to achieve
  different privacy-utility trade-offs by a user-specified hyperparameter. We
  achieve an extraction rate reduction of up to 97.7% relative to our baseline,
  with a perplexity increase of 16.9%.
paper_ai_summary: Prompt-tuning can control the extraction rates of memorized content in large language models, offering both attack and defense mechanisms to manage privacy risks.
paper_ai_keywords: prompt-tuning extraction rates memorized content LLMs GPT-Neo perplexity privacy-utility trade-offs