X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Multi-turn interactions with language models (LMs) pose critical safety
risks, as harmful intent can be strategically spread across exchanges. Yet, the
vast majority of prior work has focused on single-turn safety, while
adaptability and diversity remain among the key challenges of multi-turn
red-teaming. To address these challenges, we present X-Teaming, a scalable
framework that systematically explores how seemingly harmless interactions
escalate into harmful outcomes and generates corresponding attack scenarios.
X-Teaming employs collaborative agents for planning, attack optimization, and
verification, achieving state-of-the-art multi-turn jailbreak effectiveness and
diversity with success rates up to 98.1% across representative leading
open-weight and closed-source models. In particular, X-Teaming achieves a 96.2%
attack success rate against the latest Claude 3.7 Sonnet model, which has been
considered nearly immune to single-turn attacks. Building on X-Teaming, we
introduce XGuard-Train, an open-source multi-turn safety training dataset that
is 20x larger than the previous best resource, comprising 30K interactive
jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our
work offers essential tools and insights for mitigating sophisticated
conversational attacks, advancing the multi-turn safety of LMs.
68070197c1bd0fc00c7d90a7
Salman Rahman
admin_assigned
2025-04-22T10:54:36.037Z
625913bd5f80a3c1aad074b6
/avatars/745e4d3c916a0f0fced72ac702ff677d.svg
Salman Rahman
user
salmannyu
68070197c1bd0fc00c7d90a8
Liwei Jiang
admin_assigned
2025-04-22T10:54:42.089Z
5efe5c89ff69163f6f59e628
/avatars/c7ea475c50fcce528c9b748f279058eb.svg
Liwei Jiang
user
liweijiang
68070197c1bd0fc00c7d90a9
James Shiffer
claimed_verified
2025-04-23T08:37:32.120Z
618b5a9dcabd3c4c8e448207
https://cdn-avatars.huggingface.co/v1/production/uploads/1636522873645-618b5a9dcabd3c4c8e448207.jpeg
James S.
user
jamessucla
68070197c1bd0fc00c7d90aa
Genglin Liu
admin_assigned
2025-04-22T10:54:53.094Z
64881deb8e004bb92b0f4845
/avatars/30a1e016d469bf7eb42c713351a9f65c.svg
Genglin Liu
user
genglinliu
68070197c1bd0fc00c7d90ab
Sheriff Issaka
68070197c1bd0fc00c7d90ac
Md Rizwan Parvez
admin_assigned
2025-04-22T10:55:13.627Z
65ae1c4468139e3c42973fe4
/avatars/b065a857dd763410caadea37a2dc01c4.svg
Md Rizwan Parvez
user
mparvez
68070197c1bd0fc00c7d90ad
Hamid Palangi
admin_assigned
2025-04-22T10:55:19.780Z
6189ddd0992df2640e3e7d40
/avatars/925f2308fc412ae352b57a1b71815028.svg
Hamid Palangi
user
hamidpalangi
68070197c1bd0fc00c7d90ae
Kai-Wei Chang
admin_assigned
2025-04-22T10:55:26.346Z
60b7b9d71b90c5d07c23fbd0
https://cdn-avatars.huggingface.co/v1/production/uploads/1622653364258-noauth.jpeg
Kai-Wei Chang
user
kaiweichang
68070197c1bd0fc00c7d90af
Yejin Choi
admin_assigned
2025-04-22T10:55:32.976Z
64d42729f63b01b7f676b176
/avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
Yejin Choi
user
yejinchoinka
68070197c1bd0fc00c7d90b0
Saadia Gabriel
claimed_verified
2025-04-24T09:11:21.220Z
62229baf267ba7dda1fe9246
/avatars/26a05a333d2d8edd19c046d039bedefe.svg
Saadia Gabriel
user
skgabriel
X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Multi-turn interactions with language models (LMs) pose critical safety
risks, as harmful intent can be strategically spread across exchanges. Yet, the
vast majority of prior work has focused on single-turn safety, while
adaptability and diversity remain among the key challenges of multi-turn
red-teaming. To address these challenges, we present X-Teaming, a scalable
framework that systematically explores how seemingly harmless interactions
escalate into harmful outcomes and generates corresponding attack scenarios.
X-Teaming employs collaborative agents for planning, attack optimization, and
verification, achieving state-of-the-art multi-turn jailbreak effectiveness and
diversity with success rates up to 98.1% across representative leading
open-weight and closed-source models. In particular, X-Teaming achieves a 96.2%
attack success rate against the latest Claude 3.7 Sonnet model, which has been
considered nearly immune to single-turn attacks. Building on X-Teaming, we
introduce XGuard-Train, an open-source multi-turn safety training dataset that
is 20x larger than the previous best resource, comprising 30K interactive
jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our
work offers essential tools and insights for mitigating sophisticated
conversational attacks, advancing the multi-turn safety of LMs.
X-Teaming is a scalable framework that explores and generates multi-turn attack scenarios against language models, achieving high success rates, and XGuard-Train is a large multi-turn safety training dataset to mitigate such attacks.
multi-turn interactions language models harmful intent single-turn safety multi-turn red-teaming X-Teaming collaborative agents attack scenarios jailbreak effectiveness attack optimization verification XGuard-Train multi-turn safety training dataset conversational attacks