Lifelong Safety Alignment for Language Models
LLMs have made impressive progress, but their growing capabilities also
expose them to highly flexible jailbreaking attacks designed to bypass safety
alignment. While many existing defenses focus on known types of attacks, it is
more critical to prepare LLMs for unseen attacks that may arise during
deployment. To address this, we propose a lifelong safety alignment framework
that enables LLMs to continuously adapt to new and evolving jailbreaking
strategies. Our framework introduces a competitive setup between two
components: a Meta-Attacker, trained to actively discover novel jailbreaking
strategies, and a Defender, trained to resist them. To effectively warm up the
Meta-Attacker, we first leverage the GPT-4o API to extract key insights from a
large collection of jailbreak-related research papers. Through iterative
training, the first iteration Meta-Attacker achieves a 73% attack success rate
(ASR) on RR and a 57% transfer ASR on LAT using only single-turn attacks.
Meanwhile, the Defender progressively improves its robustness and ultimately
reduces the Meta-Attacker's success rate to just 7%, enabling safer and more
reliable deployment of LLMs in open-ended environments. The code is available
at https://github.com/sail-sg/LifelongSafetyAlignment.
6835346b2fdc5f8e8ea1e3cf
Haoyu Wang
6835346b2fdc5f8e8ea1e3d0
Zeyu Qin
6835346b2fdc5f8e8ea1e3d1
Yifei Zhao
6835346b2fdc5f8e8ea1e3d2
Chao Du
6835346b2fdc5f8e8ea1e3d3
Min Lin
6835346b2fdc5f8e8ea1e3d4
Xueqian Wang
6835346b2fdc5f8e8ea1e3d5
Tianyu Pang
admin_assigned
2025-05-27T12:31:57.554Z
63d91b6d255ef6add20e1b38
https://cdn-avatars.huggingface.co/v1/production/uploads/1675921369867-63d91b6d255ef6add20e1b38.jpeg
Tianyu Pang
user
P2333
Lifelong Safety Alignment for Language Models
LLMs have made impressive progress, but their growing capabilities also
expose them to highly flexible jailbreaking attacks designed to bypass safety
alignment. While many existing defenses focus on known types of attacks, it is
more critical to prepare LLMs for unseen attacks that may arise during
deployment. To address this, we propose a lifelong safety alignment framework
that enables LLMs to continuously adapt to new and evolving jailbreaking
strategies. Our framework introduces a competitive setup between two
components: a Meta-Attacker, trained to actively discover novel jailbreaking
strategies, and a Defender, trained to resist them. To effectively warm up the
Meta-Attacker, we first leverage the GPT-4o API to extract key insights from a
large collection of jailbreak-related research papers. Through iterative
training, the first iteration Meta-Attacker achieves a 73% attack success rate
(ASR) on RR and a 57% transfer ASR on LAT using only single-turn attacks.
Meanwhile, the Defender progressively improves its robustness and ultimately
reduces the Meta-Attacker's success rate to just 7%, enabling safer and more
reliable deployment of LLMs in open-ended environments. The code is available
at https://github.com/sail-sg/LifelongSafetyAlignment.
A lifecycle safety alignment framework employs a Meta-Attacker and Defender to adapt LLMs to novel jailbreaking strategies, improving robustness in deployment.
LLMs jailbreaking attacks safety alignment lifelong safety alignment framework Meta-Attacker Defender GPT-4o API attack success rate transfer attack success rate single-turn attacks