Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target
  Self-Distillation
This paper introduces Unilogit, a novel self-distillation method for machine
unlearning in Large Language Models. Unilogit addresses the challenge of
selectively forgetting specific information while maintaining overall model
utility, a critical task in compliance with data privacy regulations like GDPR.
Unlike prior methods that rely on static hyperparameters or starting model
outputs, Unilogit dynamically adjusts target logits to achieve a uniform
probability for the target token, leveraging the current model's outputs for
more accurate self-distillation targets. This approach not only eliminates the
need for additional hyperparameters but also enhances the model's ability to
approximate the golden targets. Extensive experiments on public benchmarks and
an in-house e-commerce dataset demonstrate Unilogit's superior performance in
balancing forget and retain objectives, outperforming state-of-the-art methods
such as NPO and UnDIAL. Our analysis further reveals Unilogit's robustness
across various scenarios, highlighting its practical applicability and
effectiveness in achieving efficacious machine unlearning.
68270512001465610c496a53
Stefan Vasilev
claimed_verified
2025-05-16T13:23:20.122Z
63a852ec353e10031a8be92e
/avatars/42ff79115c7b84fa00e7c5df373dc77d.svg
Stefan Petkov Vasilev
user
stefanvasilev
68270512001465610c496a54
Christian Herold
68270512001465610c496a55
Baohao Liao
claimed_verified
2025-07-05T07:54:18.033Z
62c414354ce7250560a1f67f
/avatars/28fd73973d1703c84f4f59644fef8a80.svg
Baohao Liao
user
baohao
68270512001465610c496a56
Seyyed Hadi Hashemi
68270512001465610c496a57
Shahram Khadivi
68270512001465610c496a58
Christof Monz
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target
  Self-Distillation
This paper introduces Unilogit, a novel self-distillation method for machine
unlearning in Large Language Models. Unilogit addresses the challenge of
selectively forgetting specific information while maintaining overall model
utility, a critical task in compliance with data privacy regulations like GDPR.
Unlike prior methods that rely on static hyperparameters or starting model
outputs, Unilogit dynamically adjusts target logits to achieve a uniform
probability for the target token, leveraging the current model's outputs for
more accurate self-distillation targets. This approach not only eliminates the
need for additional hyperparameters but also enhances the model's ability to
approximate the golden targets. Extensive experiments on public benchmarks and
an in-house e-commerce dataset demonstrate Unilogit's superior performance in
balancing forget and retain objectives, outperforming state-of-the-art methods
such as NPO and UnDIAL. Our analysis further reveals Unilogit's robustness
across various scenarios, highlighting its practical applicability and
effectiveness in achieving efficacious machine unlearning.
Unilogit dynamically adjusts logits to enable selective forgetting in Large Language Models while maintaining overall utility and outperforming existing methods.
self-distillation machine unlearning Large Language Models GDPR hyperparameters target logits golden targets NPO UnDIAL