title: Are aligned neural networks adversarially aligned?
summary: Large language models are now tuned to align with the goals of their
  creators, namely to be "helpful and harmless." These models should respond
  helpfully to user questions, but refuse to answer requests that could cause
  harm. However, adversarial users can construct inputs which circumvent attempts
  at alignment. In this work, we study to what extent these models remain
  aligned, even when interacting with an adversarial user who constructs
  worst-case inputs (adversarial examples). These inputs are designed to cause
  the model to emit harmful content that would otherwise be prohibited. We show
  that existing NLP-based optimization attacks are insufficiently powerful to
  reliably attack aligned text models: even when current NLP-based attacks fail,
  we can find adversarial inputs with brute force. As a result, the failure of
  current attacks should not be seen as proof that aligned text models remain
  aligned under adversarial inputs.
    However the recent trend in large-scale ML models is multimodal models that
  allow users to provide images that influence the text that is generated. We
  show these models can be easily attacked, i.e., induced to perform arbitrary
  un-aligned behavior through adversarial perturbation of the input image. We
  conjecture that improved NLP attacks may demonstrate this same level of
  adversarial control over text-only models.
paper_authors._id: 649b94a79fc875a8632aa083
paper_authors.name: Nicholas Carlini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-28T08:05:03.414Z
paper_authors._id: 6303fa9ba362e7e8b51d8f2a
paper_authors.avatarUrl: /avatars/53e53c84f987989deb351dd2ae6ee558.svg
paper_authors.fullname: Nicholas Carlini
paper_authors.type: user
paper_authors.user: carlini
paper_authors._id: 649b94a79fc875a8632aa084
paper_authors.name: Milad Nasr
paper_authors._id: 649b94a79fc875a8632aa085
paper_authors.name: Christopher A. Choquette-Choo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-28T20:15:55.500Z
paper_authors._id: 649c5307e89a7650158f4965
paper_authors.avatarUrl: /avatars/3c71b501db27c86ed686ae26ad1391cd.svg
paper_authors.fullname: Christopher A. Choquette-Choo
paper_authors.type: user
paper_authors.user: cchoquette
paper_authors._id: 649b94a79fc875a8632aa086
paper_authors.name: Matthew Jagielski
paper_authors._id: 649b94a79fc875a8632aa087
paper_authors.name: Irena Gao
paper_authors._id: 649b94a79fc875a8632aa088
paper_authors.name: Anas Awadalla
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-19T22:15:25.293Z
paper_authors._id: 60f1ab3afe0d78e01037eeb1
paper_authors.avatarUrl: /avatars/f784fa423fd84fffb4683fa837ffc5a3.svg
paper_authors.fullname: Anas Awadalla
paper_authors.type: user
paper_authors.user: anas-awadalla
paper_authors._id: 649b94a79fc875a8632aa089
paper_authors.name: Pang Wei Koh
paper_authors._id: 649b94a79fc875a8632aa08a
paper_authors.name: Daphne Ippolito
paper_authors._id: 649b94a79fc875a8632aa08b
paper_authors.name: Katherine Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-28T20:15:57.757Z
paper_authors._id: 612a1e924f3c2ac50a06f950
paper_authors.avatarUrl: /avatars/d424605e0d48327082114b878b91b161.svg
paper_authors.fullname: Katherine Lee
paper_authors.type: user
paper_authors.user: katherine1ee
paper_authors._id: 649b94a79fc875a8632aa08c
paper_authors.name: Florian Tramer
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-28T20:15:53.062Z
paper_authors._id: 63568f18ba90b4ea9fe91cb5
paper_authors.avatarUrl: /avatars/3e8b3c573e20cf80d329a312bfc34728.svg
paper_authors.fullname: Florian Tramer
paper_authors.type: user
paper_authors.user: ftramer
paper_authors._id: 649b94a79fc875a8632aa08d
paper_authors.name: Ludwig Schmidt
paper_title: Are aligned neural networks adversarially aligned?
paper_summary: Large language models are now tuned to align with the goals of their
  creators, namely to be "helpful and harmless." These models should respond
  helpfully to user questions, but refuse to answer requests that could cause
  harm. However, adversarial users can construct inputs which circumvent attempts
  at alignment. In this work, we study to what extent these models remain
  aligned, even when interacting with an adversarial user who constructs
  worst-case inputs (adversarial examples). These inputs are designed to cause
  the model to emit harmful content that would otherwise be prohibited. We show
  that existing NLP-based optimization attacks are insufficiently powerful to
  reliably attack aligned text models: even when current NLP-based attacks fail,
  we can find adversarial inputs with brute force. As a result, the failure of
  current attacks should not be seen as proof that aligned text models remain
  aligned under adversarial inputs.
    However the recent trend in large-scale ML models is multimodal models that
  allow users to provide images that influence the text that is generated. We
  show these models can be easily attacked, i.e., induced to perform arbitrary
  un-aligned behavior through adversarial perturbation of the input image. We
  conjecture that improved NLP attacks may demonstrate this same level of
  adversarial control over text-only models.
paper_ai_summary: Adversarial attacks on multimodal models can induce unaligned text generation, challenging the robustness of aligned large language models.
paper_ai_keywords: large language models alignment adversarial examples NLP-based optimization attacks multimodal models