title: Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
summary: We introduce Llama Guard, an LLM-based input-output safeguard model geared
  towards Human-AI conversation use cases. Our model incorporates a safety risk
  taxonomy, a valuable tool for categorizing a specific set of safety risks found
  in LLM prompts (i.e., prompt classification). This taxonomy is also
  instrumental in classifying the responses generated by LLMs to these prompts, a
  process we refer to as response classification. For the purpose of both prompt
  and response classification, we have meticulously gathered a dataset of high
  quality. Llama Guard, a Llama2-7b model that is instruction-tuned on our
  collected dataset, albeit low in volume, demonstrates strong performance on
  existing benchmarks such as the OpenAI Moderation Evaluation dataset and
  ToxicChat, where its performance matches or exceeds that of currently available
  content moderation tools. Llama Guard functions as a language model, carrying
  out multi-class classification and generating binary decision scores.
  Furthermore, the instruction fine-tuning of Llama Guard allows for the
  customization of tasks and the adaptation of output formats. This feature
  enhances the model's capabilities, such as enabling the adjustment of taxonomy
  categories to align with specific use cases, and facilitating zero-shot or
  few-shot prompting with diverse taxonomies at the input. We are making Llama
  Guard model weights available and we encourage researchers to further develop
  and adapt them to meet the evolving needs of the community for AI safety.
paper_authors._id: 6579160bdea7e2122d2cb5c3
paper_authors.name: Hakan Inan
paper_authors._id: 6579160bdea7e2122d2cb5c4
paper_authors.name: Kartikeya Upasani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T10:07:11.657Z
paper_authors._id: 65486c98bd25cef7d193c4b7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8bb5AeyaE7PBzAWVMj6pU.png
paper_authors.fullname: Kartikeya Upasani
paper_authors.type: user
paper_authors.user: litesaber
paper_authors._id: 6579160bdea7e2122d2cb5c5
paper_authors.name: Jianfeng Chi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T10:07:30.006Z
paper_authors._id: 618157cded21fa2e251905e1
paper_authors.avatarUrl: /avatars/08995325aa623c399e925b1b9986dbb3.svg
paper_authors.fullname: Jianfeng Chi
paper_authors.type: user
paper_authors.user: jianfengchi
paper_authors._id: 6579160bdea7e2122d2cb5c6
paper_authors.name: Rashi Rungta
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T10:07:37.875Z
paper_authors._id: 6227b906e78328793ae0edd9
paper_authors.avatarUrl: /avatars/7ed1773abb419d7862940005b7970d03.svg
paper_authors.fullname: Rashi Rungta
paper_authors.type: user
paper_authors.user: rr654
paper_authors._id: 6579160bdea7e2122d2cb5c7
paper_authors.name: Krithika Iyer
paper_authors._id: 6579160bdea7e2122d2cb5c8
paper_authors.name: Yuning Mao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T10:08:02.301Z
paper_authors._id: 62bf46a630a1888a761f8619
paper_authors.avatarUrl: /avatars/de6c656066b1e88d37882ce68a0fd25c.svg
paper_authors.fullname: Yuning Mao
paper_authors.type: user
paper_authors.user: yuningm
paper_authors._id: 6579160bdea7e2122d2cb5c9
paper_authors.name: Michael Tontchev
paper_authors._id: 6579160bdea7e2122d2cb5ca
paper_authors.name: Qing Hu
paper_authors._id: 6579160bdea7e2122d2cb5cb
paper_authors.name: Brian Fuller
paper_authors._id: 6579160bdea7e2122d2cb5cc
paper_authors.name: Davide Testuggine
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T10:09:16.575Z
paper_authors._id: 64e927131d8b651a9ac0bee4
paper_authors.avatarUrl: /avatars/c74a3d2b8ffd75afcad35da39f5fad51.svg
paper_authors.fullname: Davide Testuggine
paper_authors.type: user
paper_authors.user: darktex
paper_authors._id: 6579160bdea7e2122d2cb5cd
paper_authors.name: Madian Khabsa
paper_title: Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
paper_summary: We introduce Llama Guard, an LLM-based input-output safeguard model geared
  towards Human-AI conversation use cases. Our model incorporates a safety risk
  taxonomy, a valuable tool for categorizing a specific set of safety risks found
  in LLM prompts (i.e., prompt classification). This taxonomy is also
  instrumental in classifying the responses generated by LLMs to these prompts, a
  process we refer to as response classification. For the purpose of both prompt
  and response classification, we have meticulously gathered a dataset of high
  quality. Llama Guard, a Llama2-7b model that is instruction-tuned on our
  collected dataset, albeit low in volume, demonstrates strong performance on
  existing benchmarks such as the OpenAI Moderation Evaluation dataset and
  ToxicChat, where its performance matches or exceeds that of currently available
  content moderation tools. Llama Guard functions as a language model, carrying
  out multi-class classification and generating binary decision scores.
  Furthermore, the instruction fine-tuning of Llama Guard allows for the
  customization of tasks and the adaptation of output formats. This feature
  enhances the model's capabilities, such as enabling the adjustment of taxonomy
  categories to align with specific use cases, and facilitating zero-shot or
  few-shot prompting with diverse taxonomies at the input. We are making Llama
  Guard model weights available and we encourage researchers to further develop
  and adapt them to meet the evolving needs of the community for AI safety.
paper_ai_summary: Llama Guard, an LLM-based safeguard model, uses a safety risk taxonomy for prompt and response classification, demonstrating strong performance on content moderation tasks with instruction fine-tuning.
paper_ai_keywords: LLM-based input-output safeguard model safety risk taxonomy prompt classification response classification multi-class classification binary decision scores instruction fine-tuning Llama2-7b OpenAI Moderation Evaluation ToxicChat zero-shot prompting few-shot prompting