Are aligned neural networks adversarially aligned?
Large language models are now tuned to align with the goals of their
creators, namely to be "helpful and harmless." These models should respond
helpfully to user questions, but refuse to answer requests that could cause
harm. However, adversarial users can construct inputs which circumvent attempts
at alignment. In this work, we study to what extent these models remain
aligned, even when interacting with an adversarial user who constructs
worst-case inputs (adversarial examples). These inputs are designed to cause
the model to emit harmful content that would otherwise be prohibited. We show
that existing NLP-based optimization attacks are insufficiently powerful to
reliably attack aligned text models: even when current NLP-based attacks fail,
we can find adversarial inputs with brute force. As a result, the failure of
current attacks should not be seen as proof that aligned text models remain
aligned under adversarial inputs.
  However the recent trend in large-scale ML models is multimodal models that
allow users to provide images that influence the text that is generated. We
show these models can be easily attacked, i.e., induced to perform arbitrary
un-aligned behavior through adversarial perturbation of the input image. We
conjecture that improved NLP attacks may demonstrate this same level of
adversarial control over text-only models.
649b94a79fc875a8632aa083
Nicholas Carlini
admin_assigned
2023-06-28T08:05:03.414Z
6303fa9ba362e7e8b51d8f2a
/avatars/53e53c84f987989deb351dd2ae6ee558.svg
Nicholas Carlini
user
carlini
649b94a79fc875a8632aa084
Milad Nasr
649b94a79fc875a8632aa085
Christopher A. Choquette-Choo
claimed_verified
2023-06-28T20:15:55.500Z
649c5307e89a7650158f4965
/avatars/3c71b501db27c86ed686ae26ad1391cd.svg
Christopher A. Choquette-Choo
user
cchoquette
649b94a79fc875a8632aa086
Matthew Jagielski
649b94a79fc875a8632aa087
Irena Gao
649b94a79fc875a8632aa088
Anas Awadalla
claimed_verified
2024-08-19T22:15:25.293Z
60f1ab3afe0d78e01037eeb1
/avatars/f784fa423fd84fffb4683fa837ffc5a3.svg
Anas Awadalla
user
anas-awadalla
649b94a79fc875a8632aa089
Pang Wei Koh
649b94a79fc875a8632aa08a
Daphne Ippolito
649b94a79fc875a8632aa08b
Katherine Lee
claimed_verified
2023-06-28T20:15:57.757Z
612a1e924f3c2ac50a06f950
/avatars/d424605e0d48327082114b878b91b161.svg
Katherine Lee
user
katherine1ee
649b94a79fc875a8632aa08c
Florian Tramer
claimed_verified
2023-06-28T20:15:53.062Z
63568f18ba90b4ea9fe91cb5
/avatars/3e8b3c573e20cf80d329a312bfc34728.svg
Florian Tramer
user
ftramer
649b94a79fc875a8632aa08d
Ludwig Schmidt
Are aligned neural networks adversarially aligned?
Large language models are now tuned to align with the goals of their
creators, namely to be "helpful and harmless." These models should respond
helpfully to user questions, but refuse to answer requests that could cause
harm. However, adversarial users can construct inputs which circumvent attempts
at alignment. In this work, we study to what extent these models remain
aligned, even when interacting with an adversarial user who constructs
worst-case inputs (adversarial examples). These inputs are designed to cause
the model to emit harmful content that would otherwise be prohibited. We show
that existing NLP-based optimization attacks are insufficiently powerful to
reliably attack aligned text models: even when current NLP-based attacks fail,
we can find adversarial inputs with brute force. As a result, the failure of
current attacks should not be seen as proof that aligned text models remain
aligned under adversarial inputs.
  However the recent trend in large-scale ML models is multimodal models that
allow users to provide images that influence the text that is generated. We
show these models can be easily attacked, i.e., induced to perform arbitrary
un-aligned behavior through adversarial perturbation of the input image. We
conjecture that improved NLP attacks may demonstrate this same level of
adversarial control over text-only models.
Adversarial attacks on multimodal models can induce unaligned text generation, challenging the robustness of aligned large language models.
large language models alignment adversarial examples NLP-based optimization attacks multimodal models