title: Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large
    Language Models
summary: Automated red-teaming has become a crucial approach for uncovering
  vulnerabilities in large language models (LLMs). However, most existing methods
  focus on isolated safety flaws, limiting their ability to adapt to dynamic
  defenses and uncover complex vulnerabilities efficiently. To address this
  challenge, we propose Auto-RT, a reinforcement learning framework that
  automatically explores and optimizes complex attack strategies to effectively
  uncover security vulnerabilities through malicious queries. Specifically, we
  introduce two key mechanisms to reduce exploration complexity and improve
  strategy optimization: 1) Early-terminated Exploration, which accelerate
  exploration by focusing on high-potential attack strategies; and 2) Progressive
  Reward Tracking algorithm with intermediate downgrade models, which dynamically
  refine the search trajectory toward successful vulnerability exploitation.
  Extensive experiments across diverse LLMs demonstrate that, by significantly
  improving exploration efficiency and automatically optimizing attack
  strategies, Auto-RT detects a boarder range of vulnerabilities, achieving a
  faster detection speed and 16.63\% higher success rates compared to existing
  methods.
paper_authors._id: 677c948a6f370093aa2a764e
paper_authors.name: Yanjiang Liu
paper_authors._id: 677c948a6f370093aa2a764f
paper_authors.name: Shuhen Zhou
paper_authors._id: 677c948a6f370093aa2a7650
paper_authors.name: Yaojie Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-07T13:24:42.703Z
paper_authors._id: 6216496a9b34d2fb49144599
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6216496a9b34d2fb49144599/41CKA_h1Ffj3RzVabSAkm.jpeg
paper_authors.fullname: Yaojie Lu
paper_authors.type: user
paper_authors.user: luyaojie
paper_authors._id: 677c948a6f370093aa2a7651
paper_authors.name: Huijia Zhu
paper_authors._id: 677c948a6f370093aa2a7652
paper_authors.name: Weiqiang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-07T13:24:28.891Z
paper_authors._id: 65817e9ee77395a0c87c9d30
paper_authors.avatarUrl: /avatars/f19f3c10e97df7dac8d3a58f41275c4c.svg
paper_authors.fullname: weiqiang wang
paper_authors.type: user
paper_authors.user: weyl
paper_authors._id: 677c948a6f370093aa2a7653
paper_authors.name: Hongyu Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-07T13:24:22.844Z
paper_authors._id: 6711c702f858a456b4b9f3a4
paper_authors.avatarUrl: /avatars/178e9567c3111ab22717c3c0dd003a6a.svg
paper_authors.fullname: Hongyu  Lin
paper_authors.type: user
paper_authors.user: sanmusunrise
paper_authors._id: 677c948a6f370093aa2a7654
paper_authors.name: Ben He
paper_authors._id: 677c948a6f370093aa2a7655
paper_authors.name: Xianpei Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-07T13:24:12.963Z
paper_authors._id: 65e99a77e71555ed193609cf
paper_authors.avatarUrl: /avatars/38ceb127883944677665da967d17dd18.svg
paper_authors.fullname: Xianpei Han
paper_authors.type: user
paper_authors.user: xphan
paper_authors._id: 677c948a6f370093aa2a7656
paper_authors.name: Le Sun
paper_title: Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large
    Language Models
paper_summary: Automated red-teaming has become a crucial approach for uncovering
  vulnerabilities in large language models (LLMs). However, most existing methods
  focus on isolated safety flaws, limiting their ability to adapt to dynamic
  defenses and uncover complex vulnerabilities efficiently. To address this
  challenge, we propose Auto-RT, a reinforcement learning framework that
  automatically explores and optimizes complex attack strategies to effectively
  uncover security vulnerabilities through malicious queries. Specifically, we
  introduce two key mechanisms to reduce exploration complexity and improve
  strategy optimization: 1) Early-terminated Exploration, which accelerate
  exploration by focusing on high-potential attack strategies; and 2) Progressive
  Reward Tracking algorithm with intermediate downgrade models, which dynamically
  refine the search trajectory toward successful vulnerability exploitation.
  Extensive experiments across diverse LLMs demonstrate that, by significantly
  improving exploration efficiency and automatically optimizing attack
  strategies, Auto-RT detects a boarder range of vulnerabilities, achieving a
  faster detection speed and 16.63\% higher success rates compared to existing
  methods.
paper_ai_keywords: reinforcement learning reinforcement learning early-terminated exploration progressive reward tracking intermediate downgrade models attack strategies vulnerability exploitation security vulnerabilities malicious queries large language models (LLMs)