title: PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing
summary: Deploying language models (LMs) necessitates outputs to be both high-quality
  and compliant with safety guidelines. Although Inference-Time Guardrails (ITG)
  offer solutions that shift model output distributions towards compliance, we
  find that current methods struggle in balancing safety with helpfulness. ITG
  Methods that safely address non-compliant queries exhibit lower helpfulness
  while those that prioritize helpfulness compromise on safety. We refer to this
  trade-off as the guardrail tax, analogous to the alignment tax. To address
  this, we propose PrimeGuard, a novel ITG method that utilizes structured
  control flow.
    PrimeGuard routes requests to different self-instantiations of the LM with
  varying instructions, leveraging its inherent instruction-following
  capabilities and in-context learning. Our tuning-free approach dynamically
  compiles system-designer guidelines for each query. We construct and release
  safe-eval, a diverse red-team safety benchmark. Extensive evaluations
  demonstrate that PrimeGuard, without fine-tuning, overcomes the guardrail tax
  by (1) significantly increasing resistance to iterative jailbreak attacks and
  (2) achieving state-of-the-art results in safety guardrailing while (3)
  matching helpfulness scores of alignment-tuned models. Extensive evaluations
  demonstrate that PrimeGuard, without fine-tuning, outperforms all competing
  baselines and overcomes the guardrail tax by improving the fraction of safe
  responses from 61% to 97% and increasing average helpfulness scores from 4.17
  to 4.29 on the largest models, while reducing attack success rate from 100% to
  8%.
    PrimeGuard implementation is available at
  https://github.com/dynamofl/PrimeGuard and safe-eval dataset is available at
  https://huggingface.co/datasets/dynamoai/safe_eval.
paper_authors._id: 66a0d32919daa28fdecea641
paper_authors.name: Blazej Manczak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T12:23:12.545Z
paper_authors._id: 65e5ee0c0c4de934fdc6c01c
paper_authors.avatarUrl: /avatars/f5d7c72a51703171e7fade609558cd5a.svg
paper_authors.fullname: Blazej Manczak
paper_authors.type: user
paper_authors.user: blazejm
paper_authors._id: 66a0d32919daa28fdecea642
paper_authors.name: Eliott Zemour
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-24T12:23:35.543Z
paper_authors._id: 6230c0132bba3cd650640529
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6230c0132bba3cd650640529/YmKBS1JdBTQWL6Qe_kPw5.png
paper_authors.fullname: Eliott Zemour
paper_authors.type: user
paper_authors.user: eliolio
paper_authors._id: 66a0d32919daa28fdecea643
paper_authors.name: Eric Lin
paper_authors._id: 66a0d32919daa28fdecea644
paper_authors.name: Vaikkunth Mugunthan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T12:24:07.456Z
paper_authors._id: 65a029cfe969415381ca6035
paper_authors.avatarUrl: /avatars/fb825981d676dbf41ce74032c87f59cd.svg
paper_authors.fullname: Vaikkunth Mugunthan
paper_authors.type: user
paper_authors.user: vaikkunth
paper_title: PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing
paper_summary: Deploying language models (LMs) necessitates outputs to be both high-quality
  and compliant with safety guidelines. Although Inference-Time Guardrails (ITG)
  offer solutions that shift model output distributions towards compliance, we
  find that current methods struggle in balancing safety with helpfulness. ITG
  Methods that safely address non-compliant queries exhibit lower helpfulness
  while those that prioritize helpfulness compromise on safety. We refer to this
  trade-off as the guardrail tax, analogous to the alignment tax. To address
  this, we propose PrimeGuard, a novel ITG method that utilizes structured
  control flow.
    PrimeGuard routes requests to different self-instantiations of the LM with
  varying instructions, leveraging its inherent instruction-following
  capabilities and in-context learning. Our tuning-free approach dynamically
  compiles system-designer guidelines for each query. We construct and release
  safe-eval, a diverse red-team safety benchmark. Extensive evaluations
  demonstrate that PrimeGuard, without fine-tuning, overcomes the guardrail tax
  by (1) significantly increasing resistance to iterative jailbreak attacks and
  (2) achieving state-of-the-art results in safety guardrailing while (3)
  matching helpfulness scores of alignment-tuned models. Extensive evaluations
  demonstrate that PrimeGuard, without fine-tuning, outperforms all competing
  baselines and overcomes the guardrail tax by improving the fraction of safe
  responses from 61% to 97% and increasing average helpfulness scores from 4.17
  to 4.29 on the largest models, while reducing attack success rate from 100% to
  8%.
    PrimeGuard implementation is available at
  https://github.com/dynamofl/PrimeGuard and safe-eval dataset is available at
  https://huggingface.co/datasets/dynamoai/safe_eval.
paper_ai_summary: PrimeGuard is a novel Inference-Time Guardrails method that enhances both safety and helpfulness by routing queries through self-instantiations of language models with structured control flow.
paper_ai_keywords: Inference-Time Guardrails ITG PrimeGuard structured control flow self-instantiations in-context learning system-designer guidelines safe-eval safety benchmark iterative jailbreak attacks helpfulness scores alignment-tuned models