title: WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web
    Tasks
summary: Powered by a large language model (LLM), a web browsing agent operates web
  browsers in a human-like manner and offers a highly transparent path toward
  automating a wide range of everyday tasks. As web agents become increasingly
  capable and demonstrate proficiency in general browsing tasks, a critical
  question emerges: Can they go beyond general browsing to robustly handle tasks
  that are tedious and complex, or chores that humans often avoid doing
  themselves? In this paper, we introduce WebChoreArena, a new fully reproducible
  benchmark comprising 532 carefully curated tasks designed to extend the scope
  of WebArena beyond general browsing to more labor-intensive and tedious tasks.
  WebChoreArena systematically integrates three key challenges: (i) Massive
  Memory tasks requiring accurate retrieval of large amounts of information in
  the observations, (ii) Calculation tasks demanding precise mathematical
  reasoning, and (iii) Long-Term Memory tasks necessitating long-term memory
  across multiple webpages. Built on top of the fully reproducible and widely
  adopted four WebArena simulation environments, WebChoreArena ensures strict
  reproducibility and enables fair, direct comparisons with the established
  WebArena benchmark, offering key insights into agent progress. Our experimental
  results demonstrate that as LLMs evolve, represented by GPT-4o, Claude 3.7
  Sonnet, and Gemini 2.5 Pro, significant improvements in performance are
  observed on WebChoreArena. These findings suggest that WebChoreArena is
  well-suited to measure the advancement of state-of-the-art LLMs with greater
  clarity. Nevertheless, the results also indicate that even with Gemini 2.5 Pro,
  there remains substantial room for improvement compared to WebArena,
  highlighting the increased challenges posed by WebChoreArena.
paper_authors._id: 683ebebfb5052f5f8741c7f5
paper_authors.name: Atsuyuki Miyai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T09:39:31.731Z
paper_authors._id: 6527b37c0ae663e384eb1b85
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6527b37c0ae663e384eb1b85/zKWa8h6YU4BWfcitpM5Pl.png
paper_authors.fullname: Atsuyuki Miyai
paper_authors.type: user
paper_authors.user: AtsuMiyai
paper_authors._id: 683ebebfb5052f5f8741c7f6
paper_authors.name: Zaiying Zhao
paper_authors._id: 683ebebfb5052f5f8741c7f7
paper_authors.name: Kazuki Egashira
paper_authors._id: 683ebebfb5052f5f8741c7f8
paper_authors.name: Atsuki Sato
paper_authors._id: 683ebebfb5052f5f8741c7f9
paper_authors.name: Tatsumi Sunada
paper_authors._id: 683ebebfb5052f5f8741c7fa
paper_authors.name: Shota Onohara
paper_authors._id: 683ebebfb5052f5f8741c7fb
paper_authors.name: Hiromasa Yamanishi
paper_authors._id: 683ebebfb5052f5f8741c7fc
paper_authors.name: Mashiro Toyooka
paper_authors._id: 683ebebfb5052f5f8741c7fd
paper_authors.name: Kunato Nishina
paper_authors._id: 683ebebfb5052f5f8741c7fe
paper_authors.name: Ryoma Maeda
paper_authors._id: 683ebebfb5052f5f8741c7ff
paper_authors.name: Kiyoharu Aizawa
paper_authors._id: 683ebebfb5052f5f8741c800
paper_authors.name: Toshihiko Yamasaki
paper_title: WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web
    Tasks
paper_summary: Powered by a large language model (LLM), a web browsing agent operates web
  browsers in a human-like manner and offers a highly transparent path toward
  automating a wide range of everyday tasks. As web agents become increasingly
  capable and demonstrate proficiency in general browsing tasks, a critical
  question emerges: Can they go beyond general browsing to robustly handle tasks
  that are tedious and complex, or chores that humans often avoid doing
  themselves? In this paper, we introduce WebChoreArena, a new fully reproducible
  benchmark comprising 532 carefully curated tasks designed to extend the scope
  of WebArena beyond general browsing to more labor-intensive and tedious tasks.
  WebChoreArena systematically integrates three key challenges: (i) Massive
  Memory tasks requiring accurate retrieval of large amounts of information in
  the observations, (ii) Calculation tasks demanding precise mathematical
  reasoning, and (iii) Long-Term Memory tasks necessitating long-term memory
  across multiple webpages. Built on top of the fully reproducible and widely
  adopted four WebArena simulation environments, WebChoreArena ensures strict
  reproducibility and enables fair, direct comparisons with the established
  WebArena benchmark, offering key insights into agent progress. Our experimental
  results demonstrate that as LLMs evolve, represented by GPT-4o, Claude 3.7
  Sonnet, and Gemini 2.5 Pro, significant improvements in performance are
  observed on WebChoreArena. These findings suggest that WebChoreArena is
  well-suited to measure the advancement of state-of-the-art LLMs with greater
  clarity. Nevertheless, the results also indicate that even with Gemini 2.5 Pro,
  there remains substantial room for improvement compared to WebArena,
  highlighting the increased challenges posed by WebChoreArena.
paper_ai_summary: WebChoreArena, a new benchmark comprising 532 tasks, extends the scope of WebArena to more complex and tedious web browsing tasks, measuring advancements in LLM capabilities.
paper_ai_keywords: LLM Web browsing agent WebChoreArena benchmark general browsing Massive Memory tasks Calculation tasks Long-Term Memory tasks WebArena simulation environments GPT-4o Claude 3.7 Sonnet Gemini 2.5 Pro