Constitutional Classifiers: Defending against Universal Jailbreaks
  across Thousands of Hours of Red Teaming
Large language models (LLMs) are vulnerable to universal jailbreaks-prompting
strategies that systematically bypass model safeguards and enable users to
carry out harmful processes that require many model interactions, like
manufacturing illegal substances at scale. To defend against these attacks, we
introduce Constitutional Classifiers: safeguards trained on synthetic data,
generated by prompting LLMs with natural language rules (i.e., a constitution)
specifying permitted and restricted content. In over 3,000 estimated hours of
red teaming, no red teamer found a universal jailbreak that could extract
information from an early classifier-guarded LLM at a similar level of detail
to an unguarded model across most target queries. On automated evaluations,
enhanced classifiers demonstrated robust defense against held-out
domain-specific jailbreaks. These classifiers also maintain deployment
viability, with an absolute 0.38% increase in production-traffic refusals and a
23.7% inference overhead. Our work demonstrates that defending against
universal jailbreaks while maintaining practical deployment viability is
tractable.
67a04e7ab6fd93f91c65457b
Mrinank Sharma
67a04e7ab6fd93f91c65457c
Meg Tong
admin_assigned
2025-02-03T11:12:10.128Z
63272a638624baac667c8bdb
https://cdn-avatars.huggingface.co/v1/production/uploads/63272a638624baac667c8bdb/ylZ-FNT9PLhn8sBCD1wQm.png
Meg Tong
user
meg-tong
67a04e7ab6fd93f91c65457d
Jesse Mu
admin_assigned
2025-02-03T11:12:02.638Z
62301010174feb5439c42e23
/avatars/cd3c8a97823e3cbc176fef245113624f.svg
Jesse Mu
user
jayelm
67a04e7ab6fd93f91c65457e
Jerry Wei
67a04e7ab6fd93f91c65457f
Jorrit Kruthoff
67a04e7ab6fd93f91c654580
Scott Goodfriend
admin_assigned
2025-02-03T11:11:51.464Z
60ef38cdd36c6e3f5e270b5c
/avatars/a6c89092322364f35eb6051178f3fbcc.svg
Scott Goodfriend
user
sgoodfriend
67a04e7ab6fd93f91c654581
Euan Ong
claimed_verified
2025-02-03T11:07:50.780Z
64f2218f0d19f5ae05f6a807
/avatars/855d9f57b075855418e2db33a110ffed.svg
Euan Ong
user
euanong
67a04e7ab6fd93f91c654582
Alwin Peng
admin_assigned
2025-02-03T11:12:18.423Z
660ed54c6923ed21e630820d
/avatars/5c613fbff6d4d36eeaeae92296c88d2c.svg
Alwin Peng
user
Primusa
67a04e7ab6fd93f91c654583
Raj Agarwal
admin_assigned
2025-02-03T11:12:26.895Z
676158c7bedb5ba8dd41cad5
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/yLEBoauB-32KlAi7arfAE.png
Raj Agarwal
user
Raj32123
67a04e7ab6fd93f91c654584
Cem Anil
admin_assigned
2025-02-03T11:12:35.275Z
62f6b27d05ca68c0e0008549
/avatars/14d0075aa1b578cd7ee5f9e68d12e2f0.svg
Cem Anil
user
anilcem
67a04e7ab6fd93f91c654585
Amanda Askell
admin_assigned
2025-02-03T11:12:44.345Z
6764dc0c7fa1ef387f891893
/avatars/eb060a92b65877ae90c1106cfa7c4314.svg
Amanda askell
user
askeii
67a04e7ab6fd93f91c654586
Nathan Bailey
admin_assigned
2025-02-03T11:12:50.549Z
665991b987aedd2a572042e1
https://cdn-avatars.huggingface.co/v1/production/uploads/665991b987aedd2a572042e1/nWbp1n_Ps_MPTmpWJcWoz.jpeg
Nathan Bailey
user
nathanbaileyw
67a04e7ab6fd93f91c654587
Joe Benton
admin_assigned
2025-02-03T11:12:58.193Z
63124bb3f568fb0098f617c7
/avatars/6109b5c05452322246843b29b4662051.svg
Joe Benton
user
JoeJBenton
67a04e7ab6fd93f91c654588
Emma Bluemke
67a04e7ab6fd93f91c654589
Samuel R. Bowman
admin_assigned
2025-02-03T11:13:11.959Z
65b7ceed0e2951626572e25d
/avatars/5d6085ca4260d663f0ddbe632c9e746c.svg
Samuel Bowman
user
samuelpbowman
67a04e7ab6fd93f91c65458a
Eric Christiansen
admin_assigned
2025-02-03T11:13:19.490Z
64dd22e1c29ed0b051d1a5c4
/avatars/c8511325f6d8cb485382c0de40975b65.svg
Eric Christiansen
user
emchristiansen
67a04e7ab6fd93f91c65458b
Hoagy Cunningham
admin_assigned
2025-02-03T11:13:24.996Z
636146e7472131c3bc538bd8
/avatars/9db880163cc0eea796165d8bf5e2a91f.svg
Hoagy Cunningham
user
HoagyC
67a04e7ab6fd93f91c65458c
Andy Dau
admin_assigned
2025-02-03T11:13:38.325Z
656f669dec366e93ca16cf98
/avatars/84e266ede3fe45d24666e1d8e03dd94d.svg
Andy Dau
user
atadau
67a04e7ab6fd93f91c65458d
Anjali Gopal
67a04e7ab6fd93f91c65458e
Rob Gilson
67a04e7ab6fd93f91c65458f
Logan Graham
67a04e7ab6fd93f91c654590
Logan Howard
67a04e7ab6fd93f91c654591
Nimit Kalra
claimed_verified
2025-02-03T08:14:42.317Z
66fc4c692408eb3bdeba876f
/avatars/66ba18ccb95d150e66d7b6930d4eb938.svg
Nimit Kalra
user
nimitkalra
67a04e7ab6fd93f91c654592
Taesung Lee
67a04e7ab6fd93f91c654593
Kevin Lin
67a04e7ab6fd93f91c654594
Peter Lofgren
67a04e7ab6fd93f91c654595
Francesco Mosconi
67a04e7ab6fd93f91c654596
Clare O'Hara
67a04e7ab6fd93f91c654597
Catherine Olsson
67a04e7ab6fd93f91c654598
Linda Petrini
67a04e7ab6fd93f91c654599
Samir Rajani
67a04e7ab6fd93f91c65459a
Nikhil Saxena
67a04e7ab6fd93f91c65459b
Alex Silverstein
67a04e7ab6fd93f91c65459c
Tanya Singh
67a04e7ab6fd93f91c65459d
Theodore Sumers
67a04e7ab6fd93f91c65459e
Leonard Tang
67a04e7ab6fd93f91c65459f
Kevin K. Troy
67a04e7ab6fd93f91c6545a0
Constantin Weisser
67a04e7ab6fd93f91c6545a1
Ruiqi Zhong
67a04e7ab6fd93f91c6545a2
Giulio Zhou
67a04e7ab6fd93f91c6545a3
Jan Leike
67a04e7ab6fd93f91c6545a4
Jared Kaplan
67a04e7ab6fd93f91c6545a5
Ethan Perez
Constitutional Classifiers: Defending against Universal Jailbreaks
  across Thousands of Hours of Red Teaming
Large language models (LLMs) are vulnerable to universal jailbreaks-prompting
strategies that systematically bypass model safeguards and enable users to
carry out harmful processes that require many model interactions, like
manufacturing illegal substances at scale. To defend against these attacks, we
introduce Constitutional Classifiers: safeguards trained on synthetic data,
generated by prompting LLMs with natural language rules (i.e., a constitution)
specifying permitted and restricted content. In over 3,000 estimated hours of
red teaming, no red teamer found a universal jailbreak that could extract
information from an early classifier-guarded LLM at a similar level of detail
to an unguarded model across most target queries. On automated evaluations,
enhanced classifiers demonstrated robust defense against held-out
domain-specific jailbreaks. These classifiers also maintain deployment
viability, with an absolute 0.38% increase in production-traffic refusals and a
23.7% inference overhead. Our work demonstrates that defending against
universal jailbreaks while maintaining practical deployment viability is
tractable.
Constitutional Classifiers, trained on synthetic data derived from natural language rules, effectively defend large language models against universal jailbreaks with minimal deployment impact.
LLMs jailbreaks Constitutional Classifiers synthetic data natural language rules red teaming universality deployment viability inference overhead