RiOSWorld: Benchmarking the Risk of Multimodal Compter-Use Agents
With the rapid development of multimodal large language models (MLLMs), they
are increasingly deployed as autonomous computer-use agents capable of
accomplishing complex computer tasks. However, a pressing issue arises: Can the
safety risk principles designed and aligned for general MLLMs in dialogue
scenarios be effectively transferred to real-world computer-use scenarios?
Existing research on evaluating the safety risks of MLLM-based computer-use
agents suffers from several limitations: it either lacks realistic interactive
environments, or narrowly focuses on one or a few specific risk types. These
limitations ignore the complexity, variability, and diversity of real-world
environments, thereby restricting comprehensive risk evaluation for
computer-use agents. To this end, we introduce RiOSWorld, a benchmark
designed to evaluate the potential risks of MLLM-based agents during real-world
computer manipulations. Our benchmark includes 492 risky tasks spanning various
computer applications, involving web, social media, multimedia, os, email, and
office software. We categorize these risks into two major classes based on
their risk source: (i) User-originated risks and (ii) Environmental risks. For
the evaluation, we evaluate safety risks from two perspectives: (i) Risk goal
intention and (ii) Risk goal completion. Extensive experiments with multimodal
agents on RiOSWorld demonstrate that current computer-use agents
confront significant safety risks in real-world scenarios. Our findings
highlight the necessity and urgency of safety alignment for computer-use agents
in real-world computer manipulation, providing valuable insights for developing
trustworthy computer-use agents. Our benchmark is publicly available at
https://yjyddq.github.io/RiOSWorld.github.io/.
6840035d7ed0da422d1fda21
Jingyi Yang
claimed_verified
2025-06-04T08:53:32.008Z
64f73a44102fbfb26410962e
/avatars/328302a495de6a4418be835456d1d3c6.svg
jingyi Yang
user
JY-Young
6840035d7ed0da422d1fda22
Shuai Shao
6840035d7ed0da422d1fda23
Dongrui Liu
6840035d7ed0da422d1fda24
Jing Shao
RiOSWorld: Benchmarking the Risk of Multimodal Compter-Use Agents
With the rapid development of multimodal large language models (MLLMs), they
are increasingly deployed as autonomous computer-use agents capable of
accomplishing complex computer tasks. However, a pressing issue arises: Can the
safety risk principles designed and aligned for general MLLMs in dialogue
scenarios be effectively transferred to real-world computer-use scenarios?
Existing research on evaluating the safety risks of MLLM-based computer-use
agents suffers from several limitations: it either lacks realistic interactive
environments, or narrowly focuses on one or a few specific risk types. These
limitations ignore the complexity, variability, and diversity of real-world
environments, thereby restricting comprehensive risk evaluation for
computer-use agents. To this end, we introduce RiOSWorld, a benchmark
designed to evaluate the potential risks of MLLM-based agents during real-world
computer manipulations. Our benchmark includes 492 risky tasks spanning various
computer applications, involving web, social media, multimedia, os, email, and
office software. We categorize these risks into two major classes based on
their risk source: (i) User-originated risks and (ii) Environmental risks. For
the evaluation, we evaluate safety risks from two perspectives: (i) Risk goal
intention and (ii) Risk goal completion. Extensive experiments with multimodal
agents on RiOSWorld demonstrate that current computer-use agents
confront significant safety risks in real-world scenarios. Our findings
highlight the necessity and urgency of safety alignment for computer-use agents
in real-world computer manipulation, providing valuable insights for developing
trustworthy computer-use agents. Our benchmark is publicly available at
https://yjyddq.github.io/RiOSWorld.github.io/.
RIOSWorld is a benchmark for evaluating safety risks of multimodal large language models in real-world computer tasks, revealing significant risks that necessitate safety alignment.
multimodal large language models autonomous computer-use agents safety risk principles RiOSWorld risky tasks computer applications user-originated risks environmental risks risk goal intention risk goal completion