Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
We introduce Llama Guard, an LLM-based input-output safeguard model geared
towards Human-AI conversation use cases. Our model incorporates a safety risk
taxonomy, a valuable tool for categorizing a specific set of safety risks found
in LLM prompts (i.e., prompt classification). This taxonomy is also
instrumental in classifying the responses generated by LLMs to these prompts, a
process we refer to as response classification. For the purpose of both prompt
and response classification, we have meticulously gathered a dataset of high
quality. Llama Guard, a Llama2-7b model that is instruction-tuned on our
collected dataset, albeit low in volume, demonstrates strong performance on
existing benchmarks such as the OpenAI Moderation Evaluation dataset and
ToxicChat, where its performance matches or exceeds that of currently available
content moderation tools. Llama Guard functions as a language model, carrying
out multi-class classification and generating binary decision scores.
Furthermore, the instruction fine-tuning of Llama Guard allows for the
customization of tasks and the adaptation of output formats. This feature
enhances the model's capabilities, such as enabling the adjustment of taxonomy
categories to align with specific use cases, and facilitating zero-shot or
few-shot prompting with diverse taxonomies at the input. We are making Llama
Guard model weights available and we encourage researchers to further develop
and adapt them to meet the evolving needs of the community for AI safety.
6579160bdea7e2122d2cb5c3
Hakan Inan
6579160bdea7e2122d2cb5c4
Kartikeya Upasani
admin_assigned
2023-12-13T10:07:11.657Z
65486c98bd25cef7d193c4b7
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8bb5AeyaE7PBzAWVMj6pU.png
Kartikeya Upasani
user
litesaber
6579160bdea7e2122d2cb5c5
Jianfeng Chi
admin_assigned
2023-12-13T10:07:30.006Z
618157cded21fa2e251905e1
/avatars/08995325aa623c399e925b1b9986dbb3.svg
Jianfeng Chi
user
jianfengchi
6579160bdea7e2122d2cb5c6
Rashi Rungta
admin_assigned
2023-12-13T10:07:37.875Z
6227b906e78328793ae0edd9
/avatars/7ed1773abb419d7862940005b7970d03.svg
Rashi Rungta
user
rr654
6579160bdea7e2122d2cb5c7
Krithika Iyer
6579160bdea7e2122d2cb5c8
Yuning Mao
admin_assigned
2023-12-13T10:08:02.301Z
62bf46a630a1888a761f8619
/avatars/de6c656066b1e88d37882ce68a0fd25c.svg
Yuning Mao
user
yuningm
6579160bdea7e2122d2cb5c9
Michael Tontchev
6579160bdea7e2122d2cb5ca
Qing Hu
6579160bdea7e2122d2cb5cb
Brian Fuller
6579160bdea7e2122d2cb5cc
Davide Testuggine
admin_assigned
2023-12-13T10:09:16.575Z
64e927131d8b651a9ac0bee4
/avatars/c74a3d2b8ffd75afcad35da39f5fad51.svg
Davide Testuggine
user
darktex
6579160bdea7e2122d2cb5cd
Madian Khabsa
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
We introduce Llama Guard, an LLM-based input-output safeguard model geared
towards Human-AI conversation use cases. Our model incorporates a safety risk
taxonomy, a valuable tool for categorizing a specific set of safety risks found
in LLM prompts (i.e., prompt classification). This taxonomy is also
instrumental in classifying the responses generated by LLMs to these prompts, a
process we refer to as response classification. For the purpose of both prompt
and response classification, we have meticulously gathered a dataset of high
quality. Llama Guard, a Llama2-7b model that is instruction-tuned on our
collected dataset, albeit low in volume, demonstrates strong performance on
existing benchmarks such as the OpenAI Moderation Evaluation dataset and
ToxicChat, where its performance matches or exceeds that of currently available
content moderation tools. Llama Guard functions as a language model, carrying
out multi-class classification and generating binary decision scores.
Furthermore, the instruction fine-tuning of Llama Guard allows for the
customization of tasks and the adaptation of output formats. This feature
enhances the model's capabilities, such as enabling the adjustment of taxonomy
categories to align with specific use cases, and facilitating zero-shot or
few-shot prompting with diverse taxonomies at the input. We are making Llama
Guard model weights available and we encourage researchers to further develop
and adapt them to meet the evolving needs of the community for AI safety.
Llama Guard, an LLM-based safeguard model, uses a safety risk taxonomy for prompt and response classification, demonstrating strong performance on content moderation tasks with instruction fine-tuning.
LLM-based input-output safeguard model safety risk taxonomy prompt classification response classification multi-class classification binary decision scores instruction fine-tuning Llama2-7b OpenAI Moderation Evaluation ToxicChat zero-shot prompting few-shot prompting