title: Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target
    Self-Distillation
summary: This paper introduces Unilogit, a novel self-distillation method for machine
  unlearning in Large Language Models. Unilogit addresses the challenge of
  selectively forgetting specific information while maintaining overall model
  utility, a critical task in compliance with data privacy regulations like GDPR.
  Unlike prior methods that rely on static hyperparameters or starting model
  outputs, Unilogit dynamically adjusts target logits to achieve a uniform
  probability for the target token, leveraging the current model's outputs for
  more accurate self-distillation targets. This approach not only eliminates the
  need for additional hyperparameters but also enhances the model's ability to
  approximate the golden targets. Extensive experiments on public benchmarks and
  an in-house e-commerce dataset demonstrate Unilogit's superior performance in
  balancing forget and retain objectives, outperforming state-of-the-art methods
  such as NPO and UnDIAL. Our analysis further reveals Unilogit's robustness
  across various scenarios, highlighting its practical applicability and
  effectiveness in achieving efficacious machine unlearning.
paper_authors._id: 68270512001465610c496a53
paper_authors.name: Stefan Vasilev
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-16T13:23:20.122Z
paper_authors._id: 63a852ec353e10031a8be92e
paper_authors.avatarUrl: /avatars/42ff79115c7b84fa00e7c5df373dc77d.svg
paper_authors.fullname: Stefan Petkov Vasilev
paper_authors.type: user
paper_authors.user: stefanvasilev
paper_authors._id: 68270512001465610c496a54
paper_authors.name: Christian Herold
paper_authors._id: 68270512001465610c496a55
paper_authors.name: Baohao Liao
paper_authors._id: 68270512001465610c496a56
paper_authors.name: Seyyed Hadi Hashemi
paper_authors._id: 68270512001465610c496a57
paper_authors.name: Shahram Khadivi
paper_authors._id: 68270512001465610c496a58
paper_authors.name: Christof Monz
paper_title: Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target
    Self-Distillation
paper_summary: This paper introduces Unilogit, a novel self-distillation method for machine
  unlearning in Large Language Models. Unilogit addresses the challenge of
  selectively forgetting specific information while maintaining overall model
  utility, a critical task in compliance with data privacy regulations like GDPR.
  Unlike prior methods that rely on static hyperparameters or starting model
  outputs, Unilogit dynamically adjusts target logits to achieve a uniform
  probability for the target token, leveraging the current model's outputs for
  more accurate self-distillation targets. This approach not only eliminates the
  need for additional hyperparameters but also enhances the model's ability to
  approximate the golden targets. Extensive experiments on public benchmarks and
  an in-house e-commerce dataset demonstrate Unilogit's superior performance in
  balancing forget and retain objectives, outperforming state-of-the-art methods
  such as NPO and UnDIAL. Our analysis further reveals Unilogit's robustness
  across various scenarios, highlighting its practical applicability and
  effectiveness in achieving efficacious machine unlearning.
paper_ai_summary: Unilogit dynamically adjusts logits to enable selective forgetting in Large Language Models while maintaining overall utility and outperforming existing methods.
paper_ai_keywords: self-distillation machine unlearning Large Language Models GDPR hyperparameters target logits golden targets NPO UnDIAL