title: JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language
    Model
summary: Multimodal large language models (MLLMs) excel in vision-language tasks but
  also pose significant risks of generating harmful content, particularly through
  jailbreak attacks. Jailbreak attacks refer to intentional manipulations that
  bypass safety mechanisms in models, leading to the generation of inappropriate
  or unsafe content. Detecting such attacks is critical to ensuring the
  responsible deployment of MLLMs. Existing jailbreak detection methods face
  three primary challenges: (1) Many rely on model hidden states or gradients,
  limiting their applicability to white-box models, where the internal workings
  of the model are accessible; (2) They involve high computational overhead from
  uncertainty-based analysis, which limits real-time detection, and (3) They
  require fully labeled harmful datasets, which are often scarce in real-world
  settings. To address these issues, we introduce a test-time adaptive framework
  called JAILDAM. Our method leverages a memory-based approach guided by
  policy-driven unsafe knowledge representations, eliminating the need for
  explicit exposure to harmful data. By dynamically updating unsafe knowledge
  during test-time, our framework improves generalization to unseen jailbreak
  strategies while maintaining efficiency. Experiments on multiple VLM jailbreak
  benchmarks demonstrate that JAILDAM delivers state-of-the-art performance in
  harmful content detection, improving both accuracy and speed.
paper_authors._id: 67f4a80a261cb1c328d9b0a2
paper_authors.name: Yi Nian
paper_authors._id: 67f4a80a261cb1c328d9b0a3
paper_authors.name: Shenzhe Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:51:27.601Z
paper_authors._id: 6614243f67d7bfc73afc6b77
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6614243f67d7bfc73afc6b77/ifdjAb_BPraEJ2eIySk_K.jpeg
paper_authors.fullname: Shenzhe Zhu
paper_authors.type: user
paper_authors.user: Chouoftears
paper_authors._id: 67f4a80a261cb1c328d9b0a4
paper_authors.name: Yuehan Qin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:46:00.895Z
paper_authors._id: 65e56670f7daaa71c1b9ee5a
paper_authors.avatarUrl: /avatars/0ff27979dd756b6bb63818ba49fcfec5.svg
paper_authors.fullname: Yuehan Qin
paper_authors.type: user
paper_authors.user: yuehanqi
paper_authors._id: 67f4a80a261cb1c328d9b0a5
paper_authors.name: Li Li
paper_authors._id: 67f4a80a261cb1c328d9b0a6
paper_authors.name: Ziyi Wang
paper_authors._id: 67f4a80a261cb1c328d9b0a7
paper_authors.name: Chaowei Xiao
paper_authors._id: 67f4a80a261cb1c328d9b0a8
paper_authors.name: Yue Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:46:42.241Z
paper_authors._id: 65e56670f7daaa71c1b9ee5a
paper_authors.avatarUrl: /avatars/0ff27979dd756b6bb63818ba49fcfec5.svg
paper_authors.fullname: Yuehan Qin
paper_authors.type: user
paper_authors.user: yuehanqi
paper_title: JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language
    Model
paper_summary: Multimodal large language models (MLLMs) excel in vision-language tasks but
  also pose significant risks of generating harmful content, particularly through
  jailbreak attacks. Jailbreak attacks refer to intentional manipulations that
  bypass safety mechanisms in models, leading to the generation of inappropriate
  or unsafe content. Detecting such attacks is critical to ensuring the
  responsible deployment of MLLMs. Existing jailbreak detection methods face
  three primary challenges: (1) Many rely on model hidden states or gradients,
  limiting their applicability to white-box models, where the internal workings
  of the model are accessible; (2) They involve high computational overhead from
  uncertainty-based analysis, which limits real-time detection, and (3) They
  require fully labeled harmful datasets, which are often scarce in real-world
  settings. To address these issues, we introduce a test-time adaptive framework
  called JAILDAM. Our method leverages a memory-based approach guided by
  policy-driven unsafe knowledge representations, eliminating the need for
  explicit exposure to harmful data. By dynamically updating unsafe knowledge
  during test-time, our framework improves generalization to unseen jailbreak
  strategies while maintaining efficiency. Experiments on multiple VLM jailbreak
  benchmarks demonstrate that JAILDAM delivers state-of-the-art performance in
  harmful content detection, improving both accuracy and speed.
paper_ai_summary: JAILDAM is a test-time adaptive framework that detects jailbreak attacks in multimodal large language models without requiring labeled harmful data, improving accuracy and efficiency.
paper_ai_keywords: multimodal large language models jailbreak attacks safety mechanisms hidden states gradients white-box models uncertainty-based analysis memory-based approach policy-driven unsafe knowledge representations harmful content detection VLM jailbreak benchmarks