title: UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning
summary: User specifications or legal frameworks often require information to be
  removed from pretrained models, including large language models (LLMs). This
  requires deleting or "forgetting" a set of data points from an already-trained
  model, which typically degrades its performance on other data points. Thus, a
  balance must be struck between removing information and keeping the model's
  other abilities intact, with a failure to balance this trade-off leading to
  poor deletion or an unusable model. To this end, we propose UPCORE
  (Utility-Preserving Coreset Selection), a method-agnostic data selection
  framework for mitigating collateral damage during unlearning. Finding that the
  model damage is correlated with the variance of the model's representations on
  the forget set, we selectively prune the forget set to remove outliers, thereby
  minimizing model degradation after unlearning. We evaluate UPCORE across three
  standard unlearning methods consistently achieving a superior balance between
  the competing objectives of deletion efficacy and model preservation. To better
  evaluate this trade-off, we introduce a new metric, measuring the
  area-under-the-curve (AUC) across standard metrics. We find that UPCORE
  improves both standard metrics and AUC, benefitting from positive transfer
  between the coreset and pruned points while reducing negative transfer from the
  forget set to points outside of it.
paper_authors._id: 67bbe93f267aa2b537b318be
paper_authors.name: Vaidehi Patil
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T09:07:15.794Z
paper_authors._id: 64f64da90efa33bfe0a3d9ba
paper_authors.avatarUrl: /avatars/c45fb015433e46a2eeb9518910f75d35.svg
paper_authors.fullname: Vaidehi Patil
paper_authors.type: user
paper_authors.user: vaidehi99
paper_authors._id: 67bbe93f267aa2b537b318bf
paper_authors.name: Elias Stengel-Eskin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:16:49.662Z
paper_authors._id: 61781c4caf41befe8ff060e8
paper_authors.avatarUrl: /avatars/8871d7b046fc28cbc8638228da8e9737.svg
paper_authors.fullname: Elias Stengel-Eskin
paper_authors.type: user
paper_authors.user: esteng
paper_authors._id: 67bbe93f267aa2b537b318c0
paper_authors.name: Mohit Bansal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:16:57.627Z
paper_authors._id: 665d9d3a057f7c508f98c625
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/665d9d3a057f7c508f98c625/u1R9P9sJoAl4zEIcetbPy.jpeg
paper_authors.fullname: Mohit Bansal
paper_authors.type: user
paper_authors.user: mohitbansal
paper_title: UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning
paper_summary: User specifications or legal frameworks often require information to be
  removed from pretrained models, including large language models (LLMs). This
  requires deleting or "forgetting" a set of data points from an already-trained
  model, which typically degrades its performance on other data points. Thus, a
  balance must be struck between removing information and keeping the model's
  other abilities intact, with a failure to balance this trade-off leading to
  poor deletion or an unusable model. To this end, we propose UPCORE
  (Utility-Preserving Coreset Selection), a method-agnostic data selection
  framework for mitigating collateral damage during unlearning. Finding that the
  model damage is correlated with the variance of the model's representations on
  the forget set, we selectively prune the forget set to remove outliers, thereby
  minimizing model degradation after unlearning. We evaluate UPCORE across three
  standard unlearning methods consistently achieving a superior balance between
  the competing objectives of deletion efficacy and model preservation. To better
  evaluate this trade-off, we introduce a new metric, measuring the
  area-under-the-curve (AUC) across standard metrics. We find that UPCORE
  improves both standard metrics and AUC, benefitting from positive transfer
  between the coreset and pruned points while reducing negative transfer from the
  forget set to points outside of it.
paper_ai_keywords: LNGMs (Large Language Models) Transformers Adaptively failure to balance Utility-Preserving Coreset (UPCORE) Unlearning Collateral damage Variance Prune the forget set Outliers Model degradation Deletion efficacy Model preservation Area-under-the-curve (AUC) Standard unlearning methods Coreset Positive transfer Negative transfer