title: ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning
summary: Autonomous agents powered by foundation models have seen widespread adoption
  across various real-world applications. However, they remain highly vulnerable
  to malicious instructions and attacks, which can result in severe consequences
  such as privacy breaches and financial losses. More critically, existing
  guardrails for LLMs are not applicable due to the complex and dynamic nature of
  agents. To tackle these challenges, we propose ShieldAgent, the first guardrail
  agent designed to enforce explicit safety policy compliance for the action
  trajectory of other protected agents through logical reasoning. Specifically,
  ShieldAgent first constructs a safety policy model by extracting verifiable
  rules from policy documents and structuring them into a set of action-based
  probabilistic rule circuits. Given the action trajectory of the protected
  agent, ShieldAgent retrieves relevant rule circuits and generates a shielding
  plan, leveraging its comprehensive tool library and executable code for formal
  verification. In addition, given the lack of guardrail benchmarks for agents,
  we introduce ShieldAgent-Bench, a dataset with 3K safety-related pairs of agent
  instructions and action trajectories, collected via SOTA attacks across 6 web
  environments and 7 risk categories. Experiments show that ShieldAgent achieves
  SOTA on ShieldAgent-Bench and three existing benchmarks, outperforming prior
  methods by 11.3% on average with a high recall of 90.1%. Additionally,
  ShieldAgent reduces API queries by 64.7% and inference time by 58.2%,
  demonstrating its high precision and efficiency in safeguarding agents.
paper_authors._id: 67f43e4d79e20b02392d9e33
paper_authors.name: Zhaorun Chen
paper_authors._id: 67f43e4d79e20b02392d9e34
paper_authors.name: Mintong Kang
paper_authors._id: 67f43e4d79e20b02392d9e35
paper_authors.name: Bo Li
paper_title: ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning
paper_summary: Autonomous agents powered by foundation models have seen widespread adoption
  across various real-world applications. However, they remain highly vulnerable
  to malicious instructions and attacks, which can result in severe consequences
  such as privacy breaches and financial losses. More critically, existing
  guardrails for LLMs are not applicable due to the complex and dynamic nature of
  agents. To tackle these challenges, we propose ShieldAgent, the first guardrail
  agent designed to enforce explicit safety policy compliance for the action
  trajectory of other protected agents through logical reasoning. Specifically,
  ShieldAgent first constructs a safety policy model by extracting verifiable
  rules from policy documents and structuring them into a set of action-based
  probabilistic rule circuits. Given the action trajectory of the protected
  agent, ShieldAgent retrieves relevant rule circuits and generates a shielding
  plan, leveraging its comprehensive tool library and executable code for formal
  verification. In addition, given the lack of guardrail benchmarks for agents,
  we introduce ShieldAgent-Bench, a dataset with 3K safety-related pairs of agent
  instructions and action trajectories, collected via SOTA attacks across 6 web
  environments and 7 risk categories. Experiments show that ShieldAgent achieves
  SOTA on ShieldAgent-Bench and three existing benchmarks, outperforming prior
  methods by 11.3% on average with a high recall of 90.1%. Additionally,
  ShieldAgent reduces API queries by 64.7% and inference time by 58.2%,
  demonstrating its high precision and efficiency in safeguarding agents.
paper_ai_summary: ShieldAgent enforces safety policy compliance for autonomous agents using logical reasoning and probabilistic rule circuits, outperforming existing methods on benchmarks while reducing computational overhead.
paper_ai_keywords: LLMs ShieldAgent safety policy compliance action trajectory logical reasoning probabilistic rule circuits ShieldAgent-Bench API queries inference time