title: ShieldGemma: Generative AI Content Moderation Based on Gemma
summary: We present ShieldGemma, a comprehensive suite of LLM-based safety content
  moderation models built upon Gemma2. These models provide robust,
  state-of-the-art predictions of safety risks across key harm types (sexually
  explicit, dangerous content, harassment, hate speech) in both user input and
  LLM-generated output. By evaluating on both public and internal benchmarks, we
  demonstrate superior performance compared to existing models, such as Llama
  Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%).
  Additionally, we present a novel LLM-based data curation pipeline, adaptable to
  a variety of safety-related tasks and beyond. We have shown strong
  generalization performance for model trained mainly on synthetic data. By
  releasing ShieldGemma, we provide a valuable resource to the research
  community, advancing LLM safety and enabling the creation of more effective
  content moderation solutions for developers.
paper_authors._id: 66aaf8e8344279e02444b2d6
paper_authors.name: Wenjun Zeng
paper_authors._id: 66aaf8e8344279e02444b2d7
paper_authors.name: Yuchi Liu
paper_authors._id: 66aaf8e8344279e02444b2d8
paper_authors.name: Ryan Mullins
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:38:00.071Z
paper_authors._id: 65a402a53522df7a27125823
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65a402a53522df7a27125823/tVR6lZlKmjPqi7dTZBGoS.jpeg
paper_authors.fullname: Ryan Mullins
paper_authors.type: user
paper_authors.user: RyanMullins
paper_authors._id: 66aaf8e8344279e02444b2d9
paper_authors.name: Ludovic Peran
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:38:07.160Z
paper_authors._id: 60e473dcdefbf317ecd88035
paper_authors.avatarUrl: /avatars/e5c069e6cfc73ea5d3d6c2a437490cdd.svg
paper_authors.fullname: LUDOVIC PERAN
paper_authors.type: user
paper_authors.user: LudovicPeran
paper_authors._id: 66aaf8e8344279e02444b2da
paper_authors.name: Joe Fernandez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:38:13.314Z
paper_authors._id: 633faf88ac39544928829b44
paper_authors.avatarUrl: /avatars/f934c383653ffde2bd7679d6eecbedd0.svg
paper_authors.fullname: Joe Fernandez
paper_authors.type: user
paper_authors.user: MrMoseby
paper_authors._id: 66aaf8e8344279e02444b2db
paper_authors.name: Hamza Harkous
paper_authors._id: 66aaf8e8344279e02444b2dc
paper_authors.name: Karthik Narasimhan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:38:35.374Z
paper_authors._id: 62d19081fddc6f47581361f5
paper_authors.avatarUrl: /avatars/8e2a1b0f7bf48fc9bb1f0566fe085d0f.svg
paper_authors.fullname: Karthik Narasimhan
paper_authors.type: user
paper_authors.user: karthikn
paper_authors._id: 66aaf8e8344279e02444b2dd
paper_authors.name: Drew Proud
paper_authors._id: 66aaf8e8344279e02444b2de
paper_authors.name: Piyush Kumar
paper_authors._id: 66aaf8e8344279e02444b2df
paper_authors.name: Bhaktipriya Radharapu
paper_authors._id: 66aaf8e8344279e02444b2e0
paper_authors.name: Olivia Sturman
paper_authors._id: 66aaf8e8344279e02444b2e1
paper_authors.name: Oscar Wahltinez
paper_title: ShieldGemma: Generative AI Content Moderation Based on Gemma
paper_summary: We present ShieldGemma, a comprehensive suite of LLM-based safety content
  moderation models built upon Gemma2. These models provide robust,
  state-of-the-art predictions of safety risks across key harm types (sexually
  explicit, dangerous content, harassment, hate speech) in both user input and
  LLM-generated output. By evaluating on both public and internal benchmarks, we
  demonstrate superior performance compared to existing models, such as Llama
  Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%).
  Additionally, we present a novel LLM-based data curation pipeline, adaptable to
  a variety of safety-related tasks and beyond. We have shown strong
  generalization performance for model trained mainly on synthetic data. By
  releasing ShieldGemma, we provide a valuable resource to the research
  community, advancing LLM safety and enabling the creation of more effective
  content moderation solutions for developers.
paper_ai_summary: ShieldGemma, a suite of LLM-based safety content moderation models, outperforms existing solutions across various harm types and demonstrates strong generalization even when trained on synthetic data.
paper_ai_keywords: LLM-based safety content moderation models Gemma2 safety risks sexually explicit dangerous content harassment hate speech Llama Guard WildCard AU-PRC LLM-based data curation pipeline synthetic data