OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
Computer use agents are LLM-based agents that can directly interact with a
graphical user interface, by processing screenshots or accessibility trees.
While these systems are gaining popularity, their safety has been largely
overlooked, despite the fact that evaluating and understanding their potential
for harmful behavior is essential for widespread adoption. To address this gap,
we introduce OS-Harm, a new benchmark for measuring safety of computer use
agents. OS-Harm is built on top of the OSWorld environment and aims to test
models across three categories of harm: deliberate user misuse, prompt
injection attacks, and model misbehavior. To cover these cases, we create 150
tasks that span several types of safety violations (harassment, copyright
infringement, disinformation, data exfiltration, etc.) and require the agent to
interact with a variety of OS applications (email client, code editor, browser,
etc.). Moreover, we propose an automated judge to evaluate both accuracy and
safety of agents that achieves high agreement with human annotations (0.76 and
0.79 F1 score). We evaluate computer use agents based on a range of frontier
models - such as o4-mini, Claude 3.7 Sonnet, Gemini 2.5 Pro - and provide
insights into their safety. In particular, all models tend to directly comply
with many deliberate misuse queries, are relatively vulnerable to static prompt
injections, and occasionally perform unsafe actions. The OS-Harm benchmark is
available at https://github.com/tml-epfl/os-harm.
6853db4199bf39f9665c7ae5
Thomas Kuntz
6853db4199bf39f9665c7ae6
Agatha Duzan
6853db4199bf39f9665c7ae7
Hao Zhao
6853db4199bf39f9665c7ae8
Francesco Croce
6853db4199bf39f9665c7ae9
Zico Kolter
6853db4199bf39f9665c7aea
Nicolas Flammarion
6853db4199bf39f9665c7aeb
Maksym Andriushchenko
OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
Computer use agents are LLM-based agents that can directly interact with a
graphical user interface, by processing screenshots or accessibility trees.
While these systems are gaining popularity, their safety has been largely
overlooked, despite the fact that evaluating and understanding their potential
for harmful behavior is essential for widespread adoption. To address this gap,
we introduce OS-Harm, a new benchmark for measuring safety of computer use
agents. OS-Harm is built on top of the OSWorld environment and aims to test
models across three categories of harm: deliberate user misuse, prompt
injection attacks, and model misbehavior. To cover these cases, we create 150
tasks that span several types of safety violations (harassment, copyright
infringement, disinformation, data exfiltration, etc.) and require the agent to
interact with a variety of OS applications (email client, code editor, browser,
etc.). Moreover, we propose an automated judge to evaluate both accuracy and
safety of agents that achieves high agreement with human annotations (0.76 and
0.79 F1 score). We evaluate computer use agents based on a range of frontier
models - such as o4-mini, Claude 3.7 Sonnet, Gemini 2.5 Pro - and provide
insights into their safety. In particular, all models tend to directly comply
with many deliberate misuse queries, are relatively vulnerable to static prompt
injections, and occasionally perform unsafe actions. The OS-Harm benchmark is
available at https://github.com/tml-epfl/os-harm.
A new benchmark called OS-Harm measures the safety of computer use agents interacting with GUIs, evaluating their susceptibility to misuse, prompt injection attacks, and misbehavior across various safety violations and applications.
LLM-based agents OS-Harm OSWorld environment deliberate user misuse prompt injection attacks model misbehavior harassment copyright infringement disinformation data exfiltration automated judge F1 score GUI