WebGames: Challenging General-Purpose Web-Browsing AI Agents
We introduce WebGames, a comprehensive benchmark suite designed to evaluate
general-purpose web-browsing AI agents through a collection of 50+ interactive
challenges. These challenges are specifically crafted to be straightforward for
humans while systematically testing the limitations of current AI systems
across fundamental browser interactions, advanced input processing, cognitive
tasks, workflow automation, and interactive entertainment. Our framework
eliminates external dependencies through a hermetic testing environment,
ensuring reproducible evaluation with verifiable ground-truth solutions. We
evaluate leading vision-language models including GPT-4o, Claude Computer-Use,
Gemini-1.5-Pro, and Qwen2-VL against human performance. Results reveal a
substantial capability gap, with the best AI system achieving only 43.1%
success rate compared to human performance of 95.7%, highlighting fundamental
limitations in current AI systems' ability to handle common web interaction
patterns that humans find intuitive. The benchmark is publicly available at
webgames.convergence.ai, offering a lightweight, client-side implementation
that facilitates rapid evaluation cycles. Through its modular architecture and
standardized challenge specifications, WebGames provides a robust foundation
for measuring progress in development of more capable web-browsing agents.
67be8866823e790d21a2bb90
George Thomas
admin_assigned
2025-02-26T09:06:53.500Z
6529aa1460e706730575baa9
/avatars/550fac58a6ebf937a65d19a48e71eb45.svg
George Thomas
user
georgethomas
67be8866823e790d21a2bb91
Alex J. Chan
extracted_pending
2025-02-26T03:20:08.029Z
636c1e4415cd58e915bc45df
https://cdn-avatars.huggingface.co/v1/production/uploads/636c1e4415cd58e915bc45df/KnPgdPe0G5ngvXaCBua6R.jpeg
Alex J. Chan
user
XanderJC
67be8866823e790d21a2bb92
Jikun Kang
claimed_verified
2025-02-26T15:37:37.981Z
6489e10ca13f65198dc6e122
/avatars/4aa9eab488157711b2f0298ddadee2f4.svg
Kang
user
JaxonK
67be8866823e790d21a2bb93
Wenqi Wu
admin_assigned
2025-02-26T09:08:43.843Z
63a2f1dfc8a2aa5d9e85f8f6
/avatars/f2191e3a0ce92563f9bfe83283d8d966.svg
Wenqi Wu
user
BiggieW
67be8866823e790d21a2bb94
Filippos Christianos
admin_assigned
2025-02-26T09:08:23.534Z
64f46b681d337935d0495d4d
/avatars/cce5a4910617931fb13062b832e14ef8.svg
Filippos Christianos
user
semitable
67be8866823e790d21a2bb95
Fraser Greenlee
admin_assigned
2025-02-26T09:08:16.380Z
5f195784925b9863e28ad610
https://cdn-avatars.huggingface.co/v1/production/uploads/1595496291585-noauth.png
Fraser Greenlee
user
Fraser
67be8866823e790d21a2bb96
Andy Toulis
67be8866823e790d21a2bb97
Marvin Purtorab
admin_assigned
2025-02-26T09:07:42.610Z
6787c4a970c0f5272f456968
/avatars/bdfa53add57b0f0a9e4e94e24115b354.svg
Marvin Purtorab
user
comvergent-marvin
WebGames: Challenging General-Purpose Web-Browsing AI Agents
We introduce WebGames, a comprehensive benchmark suite designed to evaluate
general-purpose web-browsing AI agents through a collection of 50+ interactive
challenges. These challenges are specifically crafted to be straightforward for
humans while systematically testing the limitations of current AI systems
across fundamental browser interactions, advanced input processing, cognitive
tasks, workflow automation, and interactive entertainment. Our framework
eliminates external dependencies through a hermetic testing environment,
ensuring reproducible evaluation with verifiable ground-truth solutions. We
evaluate leading vision-language models including GPT-4o, Claude Computer-Use,
Gemini-1.5-Pro, and Qwen2-VL against human performance. Results reveal a
substantial capability gap, with the best AI system achieving only 43.1%
success rate compared to human performance of 95.7%, highlighting fundamental
limitations in current AI systems' ability to handle common web interaction
patterns that humans find intuitive. The benchmark is publicly available at
webgames.convergence.ai, offering a lightweight, client-side implementation
that facilitates rapid evaluation cycles. Through its modular architecture and
standardized challenge specifications, WebGames provides a robust foundation
for measuring progress in development of more capable web-browsing agents.
WebGames is a benchmark suite evaluating web-browsing AI agents across various browser interactions and tasks, revealing significant gaps compared to human performance.