ShieldGemma: Generative AI Content Moderation Based on Gemma
We present ShieldGemma, a comprehensive suite of LLM-based safety content
moderation models built upon Gemma2. These models provide robust,
state-of-the-art predictions of safety risks across key harm types (sexually
explicit, dangerous content, harassment, hate speech) in both user input and
LLM-generated output. By evaluating on both public and internal benchmarks, we
demonstrate superior performance compared to existing models, such as Llama
Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%).
Additionally, we present a novel LLM-based data curation pipeline, adaptable to
a variety of safety-related tasks and beyond. We have shown strong
generalization performance for model trained mainly on synthetic data. By
releasing ShieldGemma, we provide a valuable resource to the research
community, advancing LLM safety and enabling the creation of more effective
content moderation solutions for developers.
66aaf8e8344279e02444b2d6
Wenjun Zeng
66aaf8e8344279e02444b2d7
Yuchi Liu
66aaf8e8344279e02444b2d8
Ryan Mullins
admin_assigned
2024-08-02T15:38:00.071Z
65a402a53522df7a27125823
https://cdn-avatars.huggingface.co/v1/production/uploads/65a402a53522df7a27125823/tVR6lZlKmjPqi7dTZBGoS.jpeg
Ryan Mullins
user
RyanMullins
66aaf8e8344279e02444b2d9
Ludovic Peran
admin_assigned
2024-08-02T15:38:07.160Z
60e473dcdefbf317ecd88035
/avatars/e5c069e6cfc73ea5d3d6c2a437490cdd.svg
LUDOVIC PERAN
user
LudovicPeran
66aaf8e8344279e02444b2da
Joe Fernandez
admin_assigned
2024-08-02T15:38:13.314Z
633faf88ac39544928829b44
/avatars/f934c383653ffde2bd7679d6eecbedd0.svg
Joe Fernandez
user
MrMoseby
66aaf8e8344279e02444b2db
Hamza Harkous
66aaf8e8344279e02444b2dc
Karthik Narasimhan
admin_assigned
2024-08-02T15:38:35.374Z
62d19081fddc6f47581361f5
/avatars/8e2a1b0f7bf48fc9bb1f0566fe085d0f.svg
Karthik Narasimhan
user
karthikn
66aaf8e8344279e02444b2dd
Drew Proud
66aaf8e8344279e02444b2de
Piyush Kumar
66aaf8e8344279e02444b2df
Bhaktipriya Radharapu
66aaf8e8344279e02444b2e0
Olivia Sturman
66aaf8e8344279e02444b2e1
Oscar Wahltinez
ShieldGemma: Generative AI Content Moderation Based on Gemma
We present ShieldGemma, a comprehensive suite of LLM-based safety content
moderation models built upon Gemma2. These models provide robust,
state-of-the-art predictions of safety risks across key harm types (sexually
explicit, dangerous content, harassment, hate speech) in both user input and
LLM-generated output. By evaluating on both public and internal benchmarks, we
demonstrate superior performance compared to existing models, such as Llama
Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%).
Additionally, we present a novel LLM-based data curation pipeline, adaptable to
a variety of safety-related tasks and beyond. We have shown strong
generalization performance for model trained mainly on synthetic data. By
releasing ShieldGemma, we provide a valuable resource to the research
community, advancing LLM safety and enabling the creation of more effective
content moderation solutions for developers.
ShieldGemma, a suite of LLM-based safety content moderation models, outperforms existing solutions across various harm types and demonstrates strong generalization even when trained on synthetic data.
LLM-based safety content moderation models Gemma2 safety risks sexually explicit dangerous content harassment hate speech Llama Guard WildCard AU-PRC LLM-based data curation pipeline synthetic data