title: Practical Unlearning for Large Language Models
summary: While LLMs have demonstrated impressive performance across various domains
  and tasks, their security issues have become increasingly severe. Machine
  unlearning (MU) has emerged as a promising solution to address these issues by
  removing the influence of undesired data on the target model without
  compromising its utility in other aspects. MU typically assumes full access to
  the original training data to preserve utility, which is difficult to achieve
  in LLM unlearning. Existing LLM unlearning methods often assume access to data
  most affected by undesired data unlearning. However, this assumption
  underestimates the entanglement among various LLM capabilities and ignores data
  access limitations due to various issues. Moreover, these LLM unlearning
  methods do not sufficiently consider that unlearning requests in real-world
  scenarios are continuously emerging. To overcome these challenges and achieve
  practical LLM unlearning, we propose the O3 framework. The O3 framework
  includes an Out-Of-Distribution (OOD) detector to measure the similarity
  between input and unlearning data, and an Orthogonal low-rank adapter (LoRA)
  for continuously unlearning requested data. The OOD detector is trained with a
  novel contrastive entropy loss and utilizes a local-global layer-aggregated
  scoring mechanism. The orthogonal LoRA achieves parameter disentanglement among
  continual unlearning requests. During inference, our O3 framework can smartly
  decide whether and to what extent to load the unlearning LoRA based on the OOD
  detector's predictions. Notably, O3's effectiveness does not rely on any
  retained data. We conducted extensive experiments on O3 and state-of-the-art
  LLM unlearning methods across three tasks and seven datasets. The results
  indicate that O3 consistently achieves the best trade-off between unlearning
  effectiveness and utility preservation, especially when facing continuous
  unlearning requests.
paper_authors._id: 6698102224de09d10cb9eaf7
paper_authors.name: Chongyang Gao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-18T09:05:55.136Z
paper_authors._id: 63252a56f2e59af5f7ff33d1
paper_authors.avatarUrl: /avatars/d63b83ee0336b89fbccf2859c541127b.svg
paper_authors.fullname: Chongyang Gao
paper_authors.type: user
paper_authors.user: gcyzsl
paper_authors._id: 6698102224de09d10cb9eaf8
paper_authors.name: Lixu Wang
paper_authors._id: 6698102224de09d10cb9eaf9
paper_authors.name: Chenkai Weng
paper_authors._id: 6698102224de09d10cb9eafa
paper_authors.name: Xiao Wang
paper_authors._id: 6698102224de09d10cb9eafb
paper_authors.name: Qi Zhu
paper_title: Practical Unlearning for Large Language Models
paper_summary: While LLMs have demonstrated impressive performance across various domains
  and tasks, their security issues have become increasingly severe. Machine
  unlearning (MU) has emerged as a promising solution to address these issues by
  removing the influence of undesired data on the target model without
  compromising its utility in other aspects. MU typically assumes full access to
  the original training data to preserve utility, which is difficult to achieve
  in LLM unlearning. Existing LLM unlearning methods often assume access to data
  most affected by undesired data unlearning. However, this assumption
  underestimates the entanglement among various LLM capabilities and ignores data
  access limitations due to various issues. Moreover, these LLM unlearning
  methods do not sufficiently consider that unlearning requests in real-world
  scenarios are continuously emerging. To overcome these challenges and achieve
  practical LLM unlearning, we propose the O3 framework. The O3 framework
  includes an Out-Of-Distribution (OOD) detector to measure the similarity
  between input and unlearning data, and an Orthogonal low-rank adapter (LoRA)
  for continuously unlearning requested data. The OOD detector is trained with a
  novel contrastive entropy loss and utilizes a local-global layer-aggregated
  scoring mechanism. The orthogonal LoRA achieves parameter disentanglement among
  continual unlearning requests. During inference, our O3 framework can smartly
  decide whether and to what extent to load the unlearning LoRA based on the OOD
  detector's predictions. Notably, O3's effectiveness does not rely on any
  retained data. We conducted extensive experiments on O3 and state-of-the-art
  LLM unlearning methods across three tasks and seven datasets. The results
  indicate that O3 consistently achieves the best trade-off between unlearning
  effectiveness and utility preservation, especially when facing continuous
  unlearning requests.
paper_ai_keywords: Out-Of-Distribution (OOD) detector Orthogonal low-rank adapter (LoRA) contrastive entropy loss local-global layer-aggregated scoring mechanism parameter disentanglement