title: X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
summary: Multi-turn interactions with language models (LMs) pose critical safety
  risks, as harmful intent can be strategically spread across exchanges. Yet, the
  vast majority of prior work has focused on single-turn safety, while
  adaptability and diversity remain among the key challenges of multi-turn
  red-teaming. To address these challenges, we present X-Teaming, a scalable
  framework that systematically explores how seemingly harmless interactions
  escalate into harmful outcomes and generates corresponding attack scenarios.
  X-Teaming employs collaborative agents for planning, attack optimization, and
  verification, achieving state-of-the-art multi-turn jailbreak effectiveness and
  diversity with success rates up to 98.1% across representative leading
  open-weight and closed-source models. In particular, X-Teaming achieves a 96.2%
  attack success rate against the latest Claude 3.7 Sonnet model, which has been
  considered nearly immune to single-turn attacks. Building on X-Teaming, we
  introduce XGuard-Train, an open-source multi-turn safety training dataset that
  is 20x larger than the previous best resource, comprising 30K interactive
  jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our
  work offers essential tools and insights for mitigating sophisticated
  conversational attacks, advancing the multi-turn safety of LMs.
paper_authors._id: 68070197c1bd0fc00c7d90a7
paper_authors.name: Salman Rahman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:54:36.037Z
paper_authors._id: 625913bd5f80a3c1aad074b6
paper_authors.avatarUrl: /avatars/745e4d3c916a0f0fced72ac702ff677d.svg
paper_authors.fullname: Salman Rahman
paper_authors.type: user
paper_authors.user: salmannyu
paper_authors._id: 68070197c1bd0fc00c7d90a8
paper_authors.name: Liwei Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:54:42.089Z
paper_authors._id: 5efe5c89ff69163f6f59e628
paper_authors.avatarUrl: /avatars/c7ea475c50fcce528c9b748f279058eb.svg
paper_authors.fullname: Liwei Jiang
paper_authors.type: user
paper_authors.user: liweijiang
paper_authors._id: 68070197c1bd0fc00c7d90a9
paper_authors.name: James Shiffer
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:37:32.120Z
paper_authors._id: 618b5a9dcabd3c4c8e448207
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1636522873645-618b5a9dcabd3c4c8e448207.jpeg
paper_authors.fullname: scoliono
paper_authors.type: user
paper_authors.user: scoliono
paper_authors._id: 68070197c1bd0fc00c7d90aa
paper_authors.name: Genglin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:54:53.094Z
paper_authors._id: 64881deb8e004bb92b0f4845
paper_authors.avatarUrl: /avatars/30a1e016d469bf7eb42c713351a9f65c.svg
paper_authors.fullname: Genglin Liu
paper_authors.type: user
paper_authors.user: genglinliu
paper_authors._id: 68070197c1bd0fc00c7d90ab
paper_authors.name: Sheriff Issaka
paper_authors._id: 68070197c1bd0fc00c7d90ac
paper_authors.name: Md Rizwan Parvez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:55:13.627Z
paper_authors._id: 65ae1c4468139e3c42973fe4
paper_authors.avatarUrl: /avatars/b065a857dd763410caadea37a2dc01c4.svg
paper_authors.fullname: Md Rizwan Parvez
paper_authors.type: user
paper_authors.user: mparvez
paper_authors._id: 68070197c1bd0fc00c7d90ad
paper_authors.name: Hamid Palangi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:55:19.780Z
paper_authors._id: 6189ddd0992df2640e3e7d40
paper_authors.avatarUrl: /avatars/925f2308fc412ae352b57a1b71815028.svg
paper_authors.fullname: Hamid Palangi
paper_authors.type: user
paper_authors.user: hamidpalangi
paper_authors._id: 68070197c1bd0fc00c7d90ae
paper_authors.name: Kai-Wei Chang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:55:26.346Z
paper_authors._id: 60b7b9d71b90c5d07c23fbd0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1622653364258-noauth.jpeg
paper_authors.fullname: Kai-Wei Chang
paper_authors.type: user
paper_authors.user: kaiweichang
paper_authors._id: 68070197c1bd0fc00c7d90af
paper_authors.name: Yejin Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:55:32.976Z
paper_authors._id: 64d42729f63b01b7f676b176
paper_authors.avatarUrl: /avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
paper_authors.fullname: Yejin Choi
paper_authors.type: user
paper_authors.user: yejinchoinka
paper_authors._id: 68070197c1bd0fc00c7d90b0
paper_authors.name: Saadia Gabriel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T09:11:21.220Z
paper_authors._id: 62229baf267ba7dda1fe9246
paper_authors.avatarUrl: /avatars/26a05a333d2d8edd19c046d039bedefe.svg
paper_authors.fullname: Saadia Gabriel
paper_authors.type: user
paper_authors.user: skgabriel
paper_title: X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
paper_summary: Multi-turn interactions with language models (LMs) pose critical safety
  risks, as harmful intent can be strategically spread across exchanges. Yet, the
  vast majority of prior work has focused on single-turn safety, while
  adaptability and diversity remain among the key challenges of multi-turn
  red-teaming. To address these challenges, we present X-Teaming, a scalable
  framework that systematically explores how seemingly harmless interactions
  escalate into harmful outcomes and generates corresponding attack scenarios.
  X-Teaming employs collaborative agents for planning, attack optimization, and
  verification, achieving state-of-the-art multi-turn jailbreak effectiveness and
  diversity with success rates up to 98.1% across representative leading
  open-weight and closed-source models. In particular, X-Teaming achieves a 96.2%
  attack success rate against the latest Claude 3.7 Sonnet model, which has been
  considered nearly immune to single-turn attacks. Building on X-Teaming, we
  introduce XGuard-Train, an open-source multi-turn safety training dataset that
  is 20x larger than the previous best resource, comprising 30K interactive
  jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our
  work offers essential tools and insights for mitigating sophisticated
  conversational attacks, advancing the multi-turn safety of LMs.
paper_ai_summary: X-Teaming is a scalable framework that explores and generates multi-turn attack scenarios against language models, achieving high success rates, and XGuard-Train is a large multi-turn safety training dataset to mitigate such attacks.
paper_ai_keywords: multi-turn interactions language models harmful intent single-turn safety multi-turn red-teaming X-Teaming collaborative agents attack scenarios jailbreak effectiveness attack optimization verification XGuard-Train multi-turn safety training dataset conversational attacks