title: Capability-Based Scaling Laws for LLM Red-Teaming
summary: As large language models grow in capability and agency, identifying
  vulnerabilities through red-teaming becomes vital for safe deployment. However,
  traditional prompt-engineering approaches may prove ineffective once
  red-teaming turns into a weak-to-strong problem, where target models surpass
  red-teamers in capabilities. To study this shift, we frame red-teaming through
  the lens of the capability gap between attacker and target. We evaluate more
  than 500 attacker-target pairs using LLM-based jailbreak attacks that mimic
  human red-teamers across diverse families, sizes, and capability levels. Three
  strong trends emerge: (i) more capable models are better attackers, (ii) attack
  success drops sharply once the target's capability exceeds the attacker's, and
  (iii) attack success rates correlate with high performance on social science
  splits of the MMLU-Pro benchmark. From these trends, we derive a jailbreaking
  scaling law that predicts attack success for a fixed target based on
  attacker-target capability gap. These findings suggest that fixed-capability
  attackers (e.g., humans) may become ineffective against future models,
  increasingly capable open-source models amplify risks for existing systems, and
  model providers must accurately measure and control models' persuasive and
  manipulative abilities to limit their effectiveness as attackers.
paper_authors._id: 68371955b0224460143a69bf
paper_authors.name: Alexander Panfilov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T19:13:54.321Z
paper_authors._id: 645410c50b50e97d3722f1d2
paper_authors.avatarUrl: /avatars/e8e079981316b26dbf0f4ada6dcdc8c6.svg
paper_authors.fullname: Alexander Panfilov
paper_authors.type: user
paper_authors.user: kotekjedi
paper_authors._id: 68371955b0224460143a69c0
paper_authors.name: Paul Kassianik
paper_authors._id: 68371955b0224460143a69c1
paper_authors.name: Maksym Andriushchenko
paper_authors._id: 68371955b0224460143a69c2
paper_authors.name: Jonas Geiping
paper_title: Capability-Based Scaling Laws for LLM Red-Teaming
paper_summary: As large language models grow in capability and agency, identifying
  vulnerabilities through red-teaming becomes vital for safe deployment. However,
  traditional prompt-engineering approaches may prove ineffective once
  red-teaming turns into a weak-to-strong problem, where target models surpass
  red-teamers in capabilities. To study this shift, we frame red-teaming through
  the lens of the capability gap between attacker and target. We evaluate more
  than 500 attacker-target pairs using LLM-based jailbreak attacks that mimic
  human red-teamers across diverse families, sizes, and capability levels. Three
  strong trends emerge: (i) more capable models are better attackers, (ii) attack
  success drops sharply once the target's capability exceeds the attacker's, and
  (iii) attack success rates correlate with high performance on social science
  splits of the MMLU-Pro benchmark. From these trends, we derive a jailbreaking
  scaling law that predicts attack success for a fixed target based on
  attacker-target capability gap. These findings suggest that fixed-capability
  attackers (e.g., humans) may become ineffective against future models,
  increasingly capable open-source models amplify risks for existing systems, and
  model providers must accurately measure and control models' persuasive and
  manipulative abilities to limit their effectiveness as attackers.
paper_ai_summary: Red-teaming with large language models reveals that attack success drops sharply when the target model's capabilities exceed the attacker's, highlighting the need for new strategies to assess and mitigate future risks.
paper_ai_keywords: large language models red-teaming prompt-engineering LLM-based jailbreak attacks MMLU-Pro benchmark jailbreaking scaling law