Practical Unlearning for Large Language Models
While LLMs have demonstrated impressive performance across various domains
and tasks, their security issues have become increasingly severe. Machine
unlearning (MU) has emerged as a promising solution to address these issues by
removing the influence of undesired data on the target model without
compromising its utility in other aspects. MU typically assumes full access to
the original training data to preserve utility, which is difficult to achieve
in LLM unlearning. Existing LLM unlearning methods often assume access to data
most affected by undesired data unlearning. However, this assumption
underestimates the entanglement among various LLM capabilities and ignores data
access limitations due to various issues. Moreover, these LLM unlearning
methods do not sufficiently consider that unlearning requests in real-world
scenarios are continuously emerging. To overcome these challenges and achieve
practical LLM unlearning, we propose the O3 framework. The O3 framework
includes an Out-Of-Distribution (OOD) detector to measure the similarity
between input and unlearning data, and an Orthogonal low-rank adapter (LoRA)
for continuously unlearning requested data. The OOD detector is trained with a
novel contrastive entropy loss and utilizes a local-global layer-aggregated
scoring mechanism. The orthogonal LoRA achieves parameter disentanglement among
continual unlearning requests. During inference, our O3 framework can smartly
decide whether and to what extent to load the unlearning LoRA based on the OOD
detector's predictions. Notably, O3's effectiveness does not rely on any
retained data. We conducted extensive experiments on O3 and state-of-the-art
LLM unlearning methods across three tasks and seven datasets. The results
indicate that O3 consistently achieves the best trade-off between unlearning
effectiveness and utility preservation, especially when facing continuous
unlearning requests.
6698102224de09d10cb9eaf7
Chongyang Gao
claimed_verified
2024-07-18T09:05:55.136Z
63252a56f2e59af5f7ff33d1
/avatars/d63b83ee0336b89fbccf2859c541127b.svg
Chongyang Gao
user
gcyzsl
6698102224de09d10cb9eaf8
Lixu Wang
6698102224de09d10cb9eaf9
Chenkai Weng
6698102224de09d10cb9eafa
Xiao Wang
6698102224de09d10cb9eafb
Qi Zhu
Practical Unlearning for Large Language Models
While LLMs have demonstrated impressive performance across various domains
and tasks, their security issues have become increasingly severe. Machine
unlearning (MU) has emerged as a promising solution to address these issues by
removing the influence of undesired data on the target model without
compromising its utility in other aspects. MU typically assumes full access to
the original training data to preserve utility, which is difficult to achieve
in LLM unlearning. Existing LLM unlearning methods often assume access to data
most affected by undesired data unlearning. However, this assumption
underestimates the entanglement among various LLM capabilities and ignores data
access limitations due to various issues. Moreover, these LLM unlearning
methods do not sufficiently consider that unlearning requests in real-world
scenarios are continuously emerging. To overcome these challenges and achieve
practical LLM unlearning, we propose the O3 framework. The O3 framework
includes an Out-Of-Distribution (OOD) detector to measure the similarity
between input and unlearning data, and an Orthogonal low-rank adapter (LoRA)
for continuously unlearning requested data. The OOD detector is trained with a
novel contrastive entropy loss and utilizes a local-global layer-aggregated
scoring mechanism. The orthogonal LoRA achieves parameter disentanglement among
continual unlearning requests. During inference, our O3 framework can smartly
decide whether and to what extent to load the unlearning LoRA based on the OOD
detector's predictions. Notably, O3's effectiveness does not rely on any
retained data. We conducted extensive experiments on O3 and state-of-the-art
LLM unlearning methods across three tasks and seven datasets. The results
indicate that O3 consistently achieves the best trade-off between unlearning
effectiveness and utility preservation, especially when facing continuous
unlearning requests.
The O3 framework uses an OOD detector and orthogonal low-rank adapter for effective and continuous LLM unlearning without needing retailed data, achieving a balance between unlearning effectiveness and utility preservation.
machine unlearning out-of-distribution detector contrastive entropy loss orthogonal low-rank adapter parameter disentanglement continual unlearning utility preservation