title: Progent: Programmable Privilege Control for LLM Agents
summary: LLM agents are an emerging form of AI systems where large language models
  (LLMs) serve as the central component, utilizing a diverse set of tools to
  complete user-assigned tasks. Despite their great potential, LLM agents pose
  significant security risks. When interacting with the external world, they may
  encounter malicious commands from attackers, leading to the execution of
  dangerous actions. A promising way to address this is by enforcing the
  principle of least privilege: allowing only essential actions for task
  completion while blocking unnecessary ones. However, achieving this is
  challenging, as it requires covering diverse agent scenarios while preserving
  both security and utility.
    We introduce Progent, the first privilege control mechanism for LLM agents.
  At its core is a domain-specific language for flexibly expressing privilege
  control policies applied during agent execution. These policies provide
  fine-grained constraints over tool calls, deciding when tool calls are
  permissible and specifying fallbacks if they are not. This enables agent
  developers and users to craft suitable policies for their specific use cases
  and enforce them deterministically to guarantee security. Thanks to its modular
  design, integrating Progent does not alter agent internals and requires only
  minimal changes to agent implementation, enhancing its practicality and
  potential for widespread adoption. To automate policy writing, we leverage LLMs
  to generate policies based on user queries, which are then updated dynamically
  for improved security and utility. Our extensive evaluation shows that it
  enables strong security while preserving high utility across three distinct
  scenarios or benchmarks: AgentDojo, ASB, and AgentPoison. Furthermore, we
  perform an in-depth analysis, showcasing the effectiveness of its core
  components and the resilience of its automated policy generation against
  adaptive attacks.
paper_authors._id: 6800a4f9f16f9f820ed748af
paper_authors.name: Tianneng Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-22T10:13:56.123Z
paper_authors._id: 64f27f74f1b6c235aed4b904
paper_authors.avatarUrl: /avatars/1129af4fcf566b87a5ff81375376bf3a.svg
paper_authors.fullname: stneng
paper_authors.type: user
paper_authors.user: stneng
paper_authors._id: 6800a4f9f16f9f820ed748b0
paper_authors.name: Jingxuan He
paper_authors._id: 6800a4f9f16f9f820ed748b1
paper_authors.name: Zhun Wang
paper_authors._id: 6800a4f9f16f9f820ed748b2
paper_authors.name: Linyu Wu
paper_authors._id: 6800a4f9f16f9f820ed748b3
paper_authors.name: Hongwei Li
paper_authors._id: 6800a4f9f16f9f820ed748b4
paper_authors.name: Wenbo Guo
paper_authors._id: 6800a4f9f16f9f820ed748b5
paper_authors.name: Dawn Song
paper_title: Progent: Programmable Privilege Control for LLM Agents
paper_summary: LLM agents are an emerging form of AI systems where large language models
  (LLMs) serve as the central component, utilizing a diverse set of tools to
  complete user-assigned tasks. Despite their great potential, LLM agents pose
  significant security risks. When interacting with the external world, they may
  encounter malicious commands from attackers, leading to the execution of
  dangerous actions. A promising way to address this is by enforcing the
  principle of least privilege: allowing only essential actions for task
  completion while blocking unnecessary ones. However, achieving this is
  challenging, as it requires covering diverse agent scenarios while preserving
  both security and utility.
    We introduce Progent, the first privilege control mechanism for LLM agents.
  At its core is a domain-specific language for flexibly expressing privilege
  control policies applied during agent execution. These policies provide
  fine-grained constraints over tool calls, deciding when tool calls are
  permissible and specifying fallbacks if they are not. This enables agent
  developers and users to craft suitable policies for their specific use cases
  and enforce them deterministically to guarantee security. Thanks to its modular
  design, integrating Progent does not alter agent internals and requires only
  minimal changes to agent implementation, enhancing its practicality and
  potential for widespread adoption. To automate policy writing, we leverage LLMs
  to generate policies based on user queries, which are then updated dynamically
  for improved security and utility. Our extensive evaluation shows that it
  enables strong security while preserving high utility across three distinct
  scenarios or benchmarks: AgentDojo, ASB, and AgentPoison. Furthermore, we
  perform an in-depth analysis, showcasing the effectiveness of its core
  components and the resilience of its automated policy generation against
  adaptive attacks.
paper_ai_summary: Progent is a privilege control mechanism for LLM agents that enforces fine-grained tool call policies using a domain-specific language, ensuring security and utility across various scenarios.
paper_ai_keywords: LLM agents privilege control mechanism domain-specific language tool calls policy writing automated policy generation security utility AgentDojo ASB AgentPoison