title: SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large
    Language Models
summary: Integrated Speech and Large Language Models (SLMs) that can follow speech
  instructions and generate relevant text responses have gained popularity
  lately. However, the safety and robustness of these models remains largely
  unclear. In this work, we investigate the potential vulnerabilities of such
  instruction-following speech-language models to adversarial attacks and
  jailbreaking. Specifically, we design algorithms that can generate adversarial
  examples to jailbreak SLMs in both white-box and black-box attack settings
  without human involvement. Additionally, we propose countermeasures to thwart
  such jailbreaking attacks. Our models, trained on dialog data with speech
  instructions, achieve state-of-the-art performance on spoken question-answering
  task, scoring over 80% on both safety and helpfulness metrics. Despite safety
  guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs
  to adversarial perturbations and transfer attacks, with average attack success
  rates of 90% and 10% respectively when evaluated on a dataset of carefully
  designed harmful questions spanning 12 different toxic categories. However, we
  demonstrate that our proposed countermeasures reduce the attack success
  significantly.
paper_authors._id: 6644f96cbb4e1cdd58cf5589
paper_authors.name: Raghuveer Peri
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-16T12:33:34.909Z
paper_authors._id: 6645eafac2e5c9986a694cc8
paper_authors.avatarUrl: /avatars/bf3defa8a9cf5cbcd2a71386deb142eb.svg
paper_authors.fullname: Raghuveer Peri
paper_authors.type: user
paper_authors.user: rperi12
paper_authors._id: 6644f96cbb4e1cdd58cf558a
paper_authors.name: Sai Muralidhar Jayanthi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-15T20:21:07.538Z
paper_authors._id: 5faf4c4984389b139cf3b4e0
paper_authors.avatarUrl: /avatars/0fa06885e2999fcab8bc833e2211779f.svg
paper_authors.fullname: Sai Muralidhar Jayanthi
paper_authors.type: user
paper_authors.user: murali1996
paper_authors._id: 6644f96cbb4e1cdd58cf558b
paper_authors.name: Srikanth Ronanki
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-17T06:53:30.174Z
paper_authors._id: 66467366a6d12d11aa1907b7
paper_authors.avatarUrl: /avatars/58e8cfc785e94ea415575408a2e78f24.svg
paper_authors.fullname: Srikanth Ronanki
paper_authors.type: user
paper_authors.user: ronanki123
paper_authors._id: 6644f96cbb4e1cdd58cf558c
paper_authors.name: Anshu Bhatia
paper_authors._id: 6644f96cbb4e1cdd58cf558d
paper_authors.name: Karel Mundnich
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-15T20:21:36.712Z
paper_authors._id: 65b03e4dbf7d3327d066e7af
paper_authors.avatarUrl: /avatars/ad60d2f207e80f742e5aaed93281b8a8.svg
paper_authors.fullname: Karel Mundnich
paper_authors.type: user
paper_authors.user: kmundnic
paper_authors._id: 6644f96cbb4e1cdd58cf558e
paper_authors.name: Saket Dingliwal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-15T20:21:54.140Z
paper_authors._id: 60de4246264e80d074e52cd8
paper_authors.avatarUrl: /avatars/cee9c01a2456c5266cada342a98a7e03.svg
paper_authors.fullname: Dingliwal
paper_authors.type: user
paper_authors.user: Saket
paper_authors._id: 6644f96cbb4e1cdd58cf558f
paper_authors.name: Nilaksh Das
paper_authors._id: 6644f96cbb4e1cdd58cf5590
paper_authors.name: Zejiang Hou
paper_authors._id: 6644f96cbb4e1cdd58cf5591
paper_authors.name: Goeric Huybrechts
paper_authors._id: 6644f96cbb4e1cdd58cf5592
paper_authors.name: Srikanth Vishnubhotla
paper_authors._id: 6644f96cbb4e1cdd58cf5593
paper_authors.name: Daniel Garcia-Romero
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-15T20:22:34.461Z
paper_authors._id: 63af027b5833626d76714bcd
paper_authors.avatarUrl: /avatars/ab8f4266f9efd5c257597738d6ddcc6d.svg
paper_authors.fullname: Daniel Garcia-Romero
paper_authors.type: user
paper_authors.user: dgromero
paper_authors._id: 6644f96cbb4e1cdd58cf5594
paper_authors.name: Sundararajan Srinivasan
paper_authors._id: 6644f96cbb4e1cdd58cf5595
paper_authors.name: Kyu J Han
paper_authors._id: 6644f96cbb4e1cdd58cf5596
paper_authors.name: Katrin Kirchhoff
paper_title: SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large
    Language Models
paper_summary: Integrated Speech and Large Language Models (SLMs) that can follow speech
  instructions and generate relevant text responses have gained popularity
  lately. However, the safety and robustness of these models remains largely
  unclear. In this work, we investigate the potential vulnerabilities of such
  instruction-following speech-language models to adversarial attacks and
  jailbreaking. Specifically, we design algorithms that can generate adversarial
  examples to jailbreak SLMs in both white-box and black-box attack settings
  without human involvement. Additionally, we propose countermeasures to thwart
  such jailbreaking attacks. Our models, trained on dialog data with speech
  instructions, achieve state-of-the-art performance on spoken question-answering
  task, scoring over 80% on both safety and helpfulness metrics. Despite safety
  guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs
  to adversarial perturbations and transfer attacks, with average attack success
  rates of 90% and 10% respectively when evaluated on a dataset of carefully
  designed harmful questions spanning 12 different toxic categories. However, we
  demonstrate that our proposed countermeasures reduce the attack success
  significantly.
paper_ai_keywords: Integrated Speech and Large Language Models (SLMs) adversarial attacks jailbreaking white-box attack settings black-box attack settings adversarial examples countermeasures dialog data spoken question-answering task safety and helpfulness metrics adversarial perturbations transfer attacks toxic categories