Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Images
<|reference_start|>Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Images: In this paper, we introduce Selective-distillation for Class and Architecture-agnostic unleaRning (SCAR), a novel approximate unlearning method. SCAR efficiently eliminates specific information while preserving the model's test accuracy without using a retain set, which is a key component in state-of-the-art approximate unlearning algorithms. Our approach utilizes a modified Mahalanobis distance to guide the unlearning of the feature vectors of the instances to be forgotten, aligning them to the nearest wrong class distribution. Moreover, we propose a distillation-trick mechanism that distills the knowledge of the original model into the unlearning model with out-of-distribution images for retaining the original model's test performance without using any retain set. Importantly, we propose a self-forget version of SCAR that unlearns without having access to the forget set. We experimentally verified the effectiveness of our method, on three public datasets, comparing it with state-of-the-art methods. Our method obtains performance higher than methods that operate without the retain set and comparable w.r.t the best methods that rely on the retain set.<|reference_end|>
@article{bonato2024is,
    title={Is Retain Set All You Need in Machine Unlearning? Restoring Performance
  of Unlearned Models with Out-Of-Distribution Images},
    author={Jacopo Bonato, Marco Cotogni, Luigi Sabetta},
    journal={arXiv preprint arXiv:2404.12922},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.12922},
    primaryClass={cs.CV cs.AI cs.LG}
}