ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging
This paper presents the ZJUKLAB team's submission for SemEval-2025 Task 4:
Unlearning Sensitive Content from Large Language Models. This task aims to
selectively erase sensitive knowledge from large language models, avoiding both
over-forgetting and under-forgetting issues. We propose an unlearning system
that leverages Model Merging (specifically TIES-Merging), combining two
specialized models into a more balanced unlearned model. Our system achieves
competitive results, ranking second among 26 teams, with an online score of
0.944 for Task Aggregate and 0.487 for overall Aggregate. In this paper, we
also conduct local experiments and perform a comprehensive analysis of the
unlearning process, examining performance trajectories, loss dynamics, and
weight perspectives, along with several supplementary experiments, to
understand the effectiveness of our method. Furthermore, we analyze the
shortcomings of our method and evaluation metrics, emphasizing that MIA scores
and ROUGE-based metrics alone are insufficient to fully evaluate successful
unlearning. Finally, we emphasize the need for more comprehensive evaluation
methodologies and rethinking of unlearning objectives in future research. Code
is available at https://github.com/zjunlp/unlearn/tree/main/semeval25.
67e602b6dabfd9d4bbf10849
Haoming Xu
extracted_pending
2025-03-28T02:00:25.958Z
66f4bdbdc51768d9d4498d16
/avatars/0f6ded5fd9cf4e6f0b180b5aa329ea33.svg
Haoming Xu
user
HaomingXu
67e602b6dabfd9d4bbf1084a
Shuxun Wang
claimed_verified
2025-03-28T08:37:57.087Z
66d270dc5ae47374c27c9e9a
https://cdn-avatars.huggingface.co/v1/production/uploads/66d270dc5ae47374c27c9e9a/41UfzGQsFl42579N-Kbd4.jpeg
Shuxun Wang
user
Saberlve
67e602b6dabfd9d4bbf1084b
Yanqiu Zhao
67e602b6dabfd9d4bbf1084c
Yi Zhong
67e602b6dabfd9d4bbf1084d
Ziyan Jiang
67e602b6dabfd9d4bbf1084e
Ningyuan Zhao
67e602b6dabfd9d4bbf1084f
Shumin Deng
67e602b6dabfd9d4bbf10850
Huajun Chen
67e602b6dabfd9d4bbf10851
Ningyu Zhang
claimed_verified
2025-05-21T12:27:20.847Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
user
Ningyu
ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging
This paper presents the ZJUKLAB team's submission for SemEval-2025 Task 4:
Unlearning Sensitive Content from Large Language Models. This task aims to
selectively erase sensitive knowledge from large language models, avoiding both
over-forgetting and under-forgetting issues. We propose an unlearning system
that leverages Model Merging (specifically TIES-Merging), combining two
specialized models into a more balanced unlearned model. Our system achieves
competitive results, ranking second among 26 teams, with an online score of
0.944 for Task Aggregate and 0.487 for overall Aggregate. In this paper, we
also conduct local experiments and perform a comprehensive analysis of the
unlearning process, examining performance trajectories, loss dynamics, and
weight perspectives, along with several supplementary experiments, to
understand the effectiveness of our method. Furthermore, we analyze the
shortcomings of our method and evaluation metrics, emphasizing that MIA scores
and ROUGE-based metrics alone are insufficient to fully evaluate successful
unlearning. Finally, we emphasize the need for more comprehensive evaluation
methodologies and rethinking of unlearning objectives in future research. Code
is available at https://github.com/zjunlp/unlearn/tree/main/semeval25.
A team presents an unlearning system using Model Merging to selectively remove sensitive knowledge from large language models, achieving competitive results in SemEval-2025 Task 4.
Model Merging TIES-Merging unlearning sensitive knowledge over-forgetting under-forgetting performance trajectories loss dynamics weight perspectives MIA scores ROUGE-based metrics unlearning objectives