title: WebGames: Challenging General-Purpose Web-Browsing AI Agents
summary: We introduce WebGames, a comprehensive benchmark suite designed to evaluate
  general-purpose web-browsing AI agents through a collection of 50+ interactive
  challenges. These challenges are specifically crafted to be straightforward for
  humans while systematically testing the limitations of current AI systems
  across fundamental browser interactions, advanced input processing, cognitive
  tasks, workflow automation, and interactive entertainment. Our framework
  eliminates external dependencies through a hermetic testing environment,
  ensuring reproducible evaluation with verifiable ground-truth solutions. We
  evaluate leading vision-language models including GPT-4o, Claude Computer-Use,
  Gemini-1.5-Pro, and Qwen2-VL against human performance. Results reveal a
  substantial capability gap, with the best AI system achieving only 43.1%
  success rate compared to human performance of 95.7%, highlighting fundamental
  limitations in current AI systems' ability to handle common web interaction
  patterns that humans find intuitive. The benchmark is publicly available at
  webgames.convergence.ai, offering a lightweight, client-side implementation
  that facilitates rapid evaluation cycles. Through its modular architecture and
  standardized challenge specifications, WebGames provides a robust foundation
  for measuring progress in development of more capable web-browsing agents.
paper_authors._id: 67be8866823e790d21a2bb90
paper_authors.name: George Thomas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:06:53.500Z
paper_authors._id: 6529aa1460e706730575baa9
paper_authors.avatarUrl: /avatars/550fac58a6ebf937a65d19a48e71eb45.svg
paper_authors.fullname: George Thomas
paper_authors.type: user
paper_authors.user: georgethomas
paper_authors._id: 67be8866823e790d21a2bb91
paper_authors.name: Alex J. Chan
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-02-26T03:20:08.029Z
paper_authors._id: 636c1e4415cd58e915bc45df
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/636c1e4415cd58e915bc45df/KnPgdPe0G5ngvXaCBua6R.jpeg
paper_authors.fullname: Alex J. Chan
paper_authors.type: user
paper_authors.user: XanderJC
paper_authors._id: 67be8866823e790d21a2bb92
paper_authors.name: Jikun Kang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T15:37:37.981Z
paper_authors._id: 6489e10ca13f65198dc6e122
paper_authors.avatarUrl: /avatars/4aa9eab488157711b2f0298ddadee2f4.svg
paper_authors.fullname: Kang
paper_authors.type: user
paper_authors.user: JaxonK
paper_authors._id: 67be8866823e790d21a2bb93
paper_authors.name: Wenqi Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:08:43.843Z
paper_authors._id: 63a2f1dfc8a2aa5d9e85f8f6
paper_authors.avatarUrl: /avatars/f2191e3a0ce92563f9bfe83283d8d966.svg
paper_authors.fullname: Wenqi Wu
paper_authors.type: user
paper_authors.user: BiggieW
paper_authors._id: 67be8866823e790d21a2bb94
paper_authors.name: Filippos Christianos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:08:23.534Z
paper_authors._id: 64f46b681d337935d0495d4d
paper_authors.avatarUrl: /avatars/cce5a4910617931fb13062b832e14ef8.svg
paper_authors.fullname: Filippos Christianos
paper_authors.type: user
paper_authors.user: semitable
paper_authors._id: 67be8866823e790d21a2bb95
paper_authors.name: Fraser Greenlee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:08:16.380Z
paper_authors._id: 5f195784925b9863e28ad610
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1595496291585-noauth.png
paper_authors.fullname: Fraser Greenlee
paper_authors.type: user
paper_authors.user: Fraser
paper_authors._id: 67be8866823e790d21a2bb96
paper_authors.name: Andy Toulis
paper_authors._id: 67be8866823e790d21a2bb97
paper_authors.name: Marvin Purtorab
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:07:42.610Z
paper_authors._id: 6787c4a970c0f5272f456968
paper_authors.avatarUrl: /avatars/bdfa53add57b0f0a9e4e94e24115b354.svg
paper_authors.fullname: Marvin Purtorab
paper_authors.type: user
paper_authors.user: comvergent-marvin
paper_title: WebGames: Challenging General-Purpose Web-Browsing AI Agents
paper_summary: We introduce WebGames, a comprehensive benchmark suite designed to evaluate
  general-purpose web-browsing AI agents through a collection of 50+ interactive
  challenges. These challenges are specifically crafted to be straightforward for
  humans while systematically testing the limitations of current AI systems
  across fundamental browser interactions, advanced input processing, cognitive
  tasks, workflow automation, and interactive entertainment. Our framework
  eliminates external dependencies through a hermetic testing environment,
  ensuring reproducible evaluation with verifiable ground-truth solutions. We
  evaluate leading vision-language models including GPT-4o, Claude Computer-Use,
  Gemini-1.5-Pro, and Qwen2-VL against human performance. Results reveal a
  substantial capability gap, with the best AI system achieving only 43.1%
  success rate compared to human performance of 95.7%, highlighting fundamental
  limitations in current AI systems' ability to handle common web interaction
  patterns that humans find intuitive. The benchmark is publicly available at
  webgames.convergence.ai, offering a lightweight, client-side implementation
  that facilitates rapid evaluation cycles. Through its modular architecture and
  standardized challenge specifications, WebGames provides a robust foundation
  for measuring progress in development of more capable web-browsing agents.