title: Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak
    Attacks?
summary: Various jailbreak attacks have been proposed to red-team Large Language
  Models (LLMs) and revealed the vulnerable safeguards of LLMs. Besides, some
  methods are not limited to the textual modality and extend the jailbreak attack
  to Multimodal Large Language Models (MLLMs) by perturbing the visual input.
  However, the absence of a universal evaluation benchmark complicates the
  performance reproduction and fair comparison. Besides, there is a lack of
  comprehensive evaluation of closed-source state-of-the-art (SOTA) models,
  especially MLLMs, such as GPT-4V. To address these issues, this work first
  builds a comprehensive jailbreak evaluation dataset with 1445 harmful questions
  covering 11 different safety policies. Based on this dataset, extensive
  red-teaming experiments are conducted on 11 different LLMs and MLLMs, including
  both SOTA proprietary models and open-source models. We then conduct a deep
  analysis of the evaluated results and find that (1) GPT4 and GPT-4V demonstrate
  better robustness against jailbreak attacks compared to open-source LLMs and
  MLLMs. (2) Llama2 and Qwen-VL-Chat are more robust compared to other
  open-source models. (3) The transferability of visual jailbreak methods is
  relatively limited compared to textual jailbreak methods. The dataset and code
  can be found here
  https://anonymous.4open.science/r/red_teaming_gpt4-C1CE/README.md .
paper_authors._id: 660f6368adee377f87919472
paper_authors.name: Shuo Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-05T15:31:33.682Z
paper_authors._id: 648cbea3dee03837c823cbf2
paper_authors.avatarUrl: /avatars/3f8c36436a5cbff2948df099ae604418.svg
paper_authors.fullname: Shuo Chen
paper_authors.type: user
paper_authors.user: ShuoChen99
paper_authors._id: 660f6368adee377f87919473
paper_authors.name: Zhen Han
paper_authors._id: 660f6368adee377f87919474
paper_authors.name: Bailan He
paper_authors._id: 660f6368adee377f87919475
paper_authors.name: Zifeng Ding
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-03T15:14:36.007Z
paper_authors._id: 662cdcebb10f82aa4f015941
paper_authors.avatarUrl: /avatars/27381448b353b0463961a0d0b51f6ed3.svg
paper_authors.fullname: Zifeng Ding
paper_authors.type: user
paper_authors.user: Beanbagdzf
paper_authors._id: 660f6368adee377f87919476
paper_authors.name: Wenqian Yu
paper_authors._id: 660f6368adee377f87919477
paper_authors.name: Philip Torr
paper_authors._id: 660f6368adee377f87919478
paper_authors.name: Volker Tresp
paper_authors._id: 660f6368adee377f87919479
paper_authors.name: Jindong Gu
paper_title: Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak
    Attacks?
paper_summary: Various jailbreak attacks have been proposed to red-team Large Language
  Models (LLMs) and revealed the vulnerable safeguards of LLMs. Besides, some
  methods are not limited to the textual modality and extend the jailbreak attack
  to Multimodal Large Language Models (MLLMs) by perturbing the visual input.
  However, the absence of a universal evaluation benchmark complicates the
  performance reproduction and fair comparison. Besides, there is a lack of
  comprehensive evaluation of closed-source state-of-the-art (SOTA) models,
  especially MLLMs, such as GPT-4V. To address these issues, this work first
  builds a comprehensive jailbreak evaluation dataset with 1445 harmful questions
  covering 11 different safety policies. Based on this dataset, extensive
  red-teaming experiments are conducted on 11 different LLMs and MLLMs, including
  both SOTA proprietary models and open-source models. We then conduct a deep
  analysis of the evaluated results and find that (1) GPT4 and GPT-4V demonstrate
  better robustness against jailbreak attacks compared to open-source LLMs and
  MLLMs. (2) Llama2 and Qwen-VL-Chat are more robust compared to other
  open-source models. (3) The transferability of visual jailbreak methods is
  relatively limited compared to textual jailbreak methods. The dataset and code
  can be found here
  https://anonymous.4open.science/r/red_teaming_gpt4-C1CE/README.md .
paper_ai_summary: A comprehensive jailbreak evaluation dataset is created to assess the robustness of various LLMs and MLLMs against jailbreak attacks, revealing differences in robustness between open-source and proprietary models, particularly in visual versus textual attacks.
paper_ai_keywords: Large Language Models Multimodal Large Language Models jailbreak attacks safety policies visual jailbreak textual jailbreak robustness transferability open-source models proprietary models