Intrinsic Evaluation of Unlearning Using Parametric Knowledge Traces
The task of "unlearning" certain concepts in large language models (LLMs) has
attracted immense attention recently, due to its importance for mitigating
undesirable model behaviours, such as the generation of harmful, private, or
incorrect information. Current protocols to evaluate unlearning methods largely
rely on behavioral tests, without monitoring the presence of unlearned
knowledge within the model's parameters. This residual knowledge can be
adversarially exploited to recover the erased information post-unlearning. We
argue that unlearning should also be evaluated internally, by considering
changes in the parametric knowledge traces of the unlearned concepts. To this
end, we propose a general methodology for eliciting directions in the parameter
space (termed "concept vectors") that encode concrete concepts, and construct
ConceptVectors, a benchmark dataset containing hundreds of common concepts and
their parametric knowledge traces within two open-source LLMs. Evaluation on
ConceptVectors shows that existing unlearning methods minimally impact concept
vectors, while directly ablating these vectors demonstrably removes the
associated knowledge from the LLMs and significantly reduces their
susceptibility to adversarial manipulation. Our results highlight limitations
in behavioral-based unlearning evaluations and call for future work to include
parametric-based evaluations. To support this, we release our code and
benchmark at https://github.com/yihuaihong/ConceptVectors.
66746362d812e2028ffaaef0
Yihuai Hong
extracted_pending
2024-06-20T17:14:10.838Z
64fe727b2a6d4e23be4db45a
/avatars/57e8f9801661c53187890c904adc2d81.svg
YihuaiHong
user
YihuaiHong
66746362d812e2028ffaaef1
Lei Yu
66746362d812e2028ffaaef2
Shauli Ravfogel
claimed_verified
2024-06-25T13:34:20.761Z
635686ec5aeb69011c7d1abd
/avatars/c59034ad2c9c2daf4b4a8d3c56449f5e.svg
Shauli Ravfogel
user
ravfogs
66746362d812e2028ffaaef3
Haiqin Yang
66746362d812e2028ffaaef4
Mor Geva
claimed_verified
2024-06-20T21:03:24.886Z
610b729f9da682cd54ad9adf
https://cdn-avatars.huggingface.co/v1/production/uploads/1628140189042-noauth.jpeg
Mor Geva
user
mega
Intrinsic Evaluation of Unlearning Using Parametric Knowledge Traces
The task of "unlearning" certain concepts in large language models (LLMs) has
attracted immense attention recently, due to its importance for mitigating
undesirable model behaviours, such as the generation of harmful, private, or
incorrect information. Current protocols to evaluate unlearning methods largely
rely on behavioral tests, without monitoring the presence of unlearned
knowledge within the model's parameters. This residual knowledge can be
adversarially exploited to recover the erased information post-unlearning. We
argue that unlearning should also be evaluated internally, by considering
changes in the parametric knowledge traces of the unlearned concepts. To this
end, we propose a general methodology for eliciting directions in the parameter
space (termed "concept vectors") that encode concrete concepts, and construct
ConceptVectors, a benchmark dataset containing hundreds of common concepts and
their parametric knowledge traces within two open-source LLMs. Evaluation on
ConceptVectors shows that existing unlearning methods minimally impact concept
vectors, while directly ablating these vectors demonstrably removes the
associated knowledge from the LLMs and significantly reduces their
susceptibility to adversarial manipulation. Our results highlight limitations
in behavioral-based unlearning evaluations and call for future work to include
parametric-based evaluations. To support this, we release our code and
benchmark at https://github.com/yihuaihong/ConceptVectors.
Evaluating concept unlearning in large language models by analyzing parametric knowledge traces reveals limitations in behavioral evaluations and suggests removing specific concept vectors to improve unlearning effectiveness.
large language models unlearning concept vectors parametric knowledge traces behavioral tests adversarial manipulation