title: Unveiling Safety Vulnerabilities of Large Language Models
summary: As large language models become more prevalent, their possible harmful or
  inappropriate responses are a cause for concern. This paper introduces a unique
  dataset containing adversarial examples in the form of questions, which we call
  AttaQ, designed to provoke such harmful or inappropriate responses. We assess
  the efficacy of our dataset by analyzing the vulnerabilities of various models
  when subjected to it. Additionally, we introduce a novel automatic approach for
  identifying and naming vulnerable semantic regions - input semantic areas for
  which the model is likely to produce harmful outputs. This is achieved through
  the application of specialized clustering techniques that consider both the
  semantic similarity of the input attacks and the harmfulness of the model's
  responses. Automatically identifying vulnerable semantic regions enhances the
  evaluation of model weaknesses, facilitating targeted improvements to its
  safety mechanisms and overall reliability.
paper_authors._id: 654afb307ed5797e22f36c01
paper_authors.name: George Kour
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:12:02.482Z
paper_authors._id: 64540de813dba495b2e0354e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64540de813dba495b2e0354e/w87XdCBRpNw6Kd2xn4AkY.png
paper_authors.fullname: George Kour
paper_authors.type: user
paper_authors.user: gkour
paper_authors._id: 654afb307ed5797e22f36c02
paper_authors.name: Marcel Zalmanovici
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:12:15.164Z
paper_authors._id: 64bfc29c5ce3d382c05bbe8f
paper_authors.avatarUrl: /avatars/5f4e6659350e20a27d19b0fb4e110bc9.svg
paper_authors.fullname: Marcel Zalmanovici
paper_authors.type: user
paper_authors.user: m-ibm
paper_authors._id: 654afb307ed5797e22f36c03
paper_authors.name: Naama Zwerdling
paper_authors._id: 654afb307ed5797e22f36c04
paper_authors.name: Esther Goldbraich
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:12:25.672Z
paper_authors._id: 62a5e10bb865939649124ae8
paper_authors.avatarUrl: /avatars/3e22777365ced0f37dfb675a934d71ec.svg
paper_authors.fullname: Esther Goldbraich
paper_authors.type: user
paper_authors.user: esthergold
paper_authors._id: 654afb307ed5797e22f36c05
paper_authors.name: Ora Nova Fandina
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-08T15:39:52.356Z
paper_authors._id: 648c5091a47a0850f947e7c3
paper_authors.avatarUrl: /avatars/580eeee0e0a07ea10a0d8bb87f5a829c.svg
paper_authors.fullname: Ora Fandina
paper_authors.type: user
paper_authors.user: orafandina
paper_authors._id: 654afb307ed5797e22f36c06
paper_authors.name: Ateret Anaby-Tavor
paper_authors._id: 654afb307ed5797e22f36c07
paper_authors.name: Orna Raz
paper_authors._id: 654afb307ed5797e22f36c08
paper_authors.name: Eitan Farchi
paper_title: Unveiling Safety Vulnerabilities of Large Language Models
paper_summary: As large language models become more prevalent, their possible harmful or
  inappropriate responses are a cause for concern. This paper introduces a unique
  dataset containing adversarial examples in the form of questions, which we call
  AttaQ, designed to provoke such harmful or inappropriate responses. We assess
  the efficacy of our dataset by analyzing the vulnerabilities of various models
  when subjected to it. Additionally, we introduce a novel automatic approach for
  identifying and naming vulnerable semantic regions - input semantic areas for
  which the model is likely to produce harmful outputs. This is achieved through
  the application of specialized clustering techniques that consider both the
  semantic similarity of the input attacks and the harmfulness of the model's
  responses. Automatically identifying vulnerable semantic regions enhances the
  evaluation of model weaknesses, facilitating targeted improvements to its
  safety mechanisms and overall reliability.
paper_ai_summary: A dataset called AttaQ is introduced to identify and evaluate vulnerable semantic regions in large language models, using adversarial questions to detect harmful responses and improve model safety.
paper_ai_keywords: adversarial examples AttaQ automatic approach vulnerable semantic regions specialized clustering techniques semantic similarity harmful outputs large language models evaluation of model weaknesses targeted improvements safety mechanisms