title: Large Language Model Unlearning via Embedding-Corrupted Prompts
summary: Large language models (LLMs) have advanced to encompass extensive knowledge
  across diverse domains. Yet controlling what a large language model should not
  know is important for ensuring alignment and thus safe use. However, accurately
  and efficiently unlearning knowledge from an LLM remains challenging due to the
  potential collateral damage caused by the fuzzy boundary between retention and
  forgetting, and the large computational requirements for optimization across
  state-of-the-art models with hundreds of billions of parameters. In this work,
  we present Embedding-COrrupted (ECO) Prompts, a lightweight unlearning
  framework for large language models to address both the challenges of knowledge
  entanglement and unlearning efficiency. Instead of relying on the LLM itself to
  unlearn, we enforce an unlearned state during inference by employing a prompt
  classifier to identify and safeguard prompts to forget. We learn corruptions
  added to prompt embeddings via zeroth order optimization toward the unlearning
  objective offline and corrupt prompts flagged by the classifier during
  inference. We find that these embedding-corrupted prompts not only lead to
  desirable outputs that satisfy the unlearning objective but also closely
  approximate the output from a model that has never been trained on the data
  intended for forgetting. Through extensive experiments on unlearning, we
  demonstrate the superiority of our method in achieving promising unlearning at
  nearly zero side effects in general domains and domains closely related to the
  unlearned ones. Additionally, we highlight the scalability of our method to 100
  LLMs, ranging from 0.5B to 236B parameters, incurring no additional cost as the
  number of parameters increases.
paper_authors._id: 666a59e2f0d87d9c3b1de24b
paper_authors.name: Chris Yuhao Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-13T09:31:07.612Z
paper_authors._id: 658229ef5f6d83438257fce5
paper_authors.avatarUrl: /avatars/b4417de9a338e95dc69cc547a46348e8.svg
paper_authors.fullname: Chris (Yuhao) Liu
paper_authors.type: user
paper_authors.user: chrisliu298
paper_authors._id: 666a59e2f0d87d9c3b1de24c
paper_authors.name: Yaxuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-13T20:27:12.286Z
paper_authors._id: 6537dbf4ca70ffcd7234a012
paper_authors.avatarUrl: /avatars/82c9490e3b36667d90cfb52b925b9c92.svg
paper_authors.fullname: Yaxuan Wang
paper_authors.type: user
paper_authors.user: Yaxuan11
paper_authors._id: 666a59e2f0d87d9c3b1de24d
paper_authors.name: Jeffrey Flanigan
paper_authors._id: 666a59e2f0d87d9c3b1de24e
paper_authors.name: Yang Liu
paper_title: Large Language Model Unlearning via Embedding-Corrupted Prompts
paper_summary: Large language models (LLMs) have advanced to encompass extensive knowledge
  across diverse domains. Yet controlling what a large language model should not
  know is important for ensuring alignment and thus safe use. However, accurately
  and efficiently unlearning knowledge from an LLM remains challenging due to the
  potential collateral damage caused by the fuzzy boundary between retention and
  forgetting, and the large computational requirements for optimization across
  state-of-the-art models with hundreds of billions of parameters. In this work,
  we present Embedding-COrrupted (ECO) Prompts, a lightweight unlearning
  framework for large language models to address both the challenges of knowledge
  entanglement and unlearning efficiency. Instead of relying on the LLM itself to
  unlearn, we enforce an unlearned state during inference by employing a prompt
  classifier to identify and safeguard prompts to forget. We learn corruptions
  added to prompt embeddings via zeroth order optimization toward the unlearning
  objective offline and corrupt prompts flagged by the classifier during
  inference. We find that these embedding-corrupted prompts not only lead to
  desirable outputs that satisfy the unlearning objective but also closely
  approximate the output from a model that has never been trained on the data
  intended for forgetting. Through extensive experiments on unlearning, we
  demonstrate the superiority of our method in achieving promising unlearning at
  nearly zero side effects in general domains and domains closely related to the
  unlearned ones. Additionally, we highlight the scalability of our method to 100
  LLMs, ranging from 0.5B to 236B parameters, incurring no additional cost as the
  number of parameters increases.
paper_ai_summary: A lightweight unlearning framework using embedding-corrupted prompts addresses knowledge entanglement and efficiency challenges in large language models, enabling effective and side-effect-free unlearning.
paper_ai_keywords: large language models unlearning prompt classifier embedding-corrupted prompts zeroth order optimization scalable unlearning