title: Explorer: Scaling Exploration-driven Web Trajectory Synthesis for
    Multimodal Web Agents
summary: Recent success in large multimodal models (LMMs) has sparked promising
  applications of agents capable of autonomously completing complex web tasks.
  While open-source LMM agents have made significant advances in offline
  evaluation benchmarks, their performance still falls substantially short of
  human-level capabilities in more realistic online settings. A key bottleneck is
  the lack of diverse and large-scale trajectory-level datasets across various
  domains, which are expensive to collect. In this paper, we address this
  challenge by developing a scalable recipe to synthesize the largest and most
  diverse trajectory-level dataset to date, containing over 94K successful
  multimodal web trajectories, spanning 49K unique URLs, 720K screenshots, and
  33M web elements. In particular, we leverage extensive web exploration and
  refinement to obtain diverse task intents. The average cost is 28 cents per
  successful trajectory, making it affordable to a wide range of users in the
  community. Leveraging this dataset, we train Explorer, a multimodal web agent,
  and demonstrate strong performance on both offline and online web agent
  benchmarks such as Mind2Web-Live, Multimodal-Mind2Web, and MiniWob++.
  Additionally, our experiments highlight data scaling as a key driver for
  improving web agent capabilities. We hope this study makes state-of-the-art
  LMM-based agent research at a larger scale more accessible.
paper_authors._id: 67b3f1f1f5bd60d66133e1f3
paper_authors.name: Vardaan Pahuja
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:31:47.969Z
paper_authors._id: 6556717676fe5cfa6a115405
paper_authors.avatarUrl: /avatars/570dd8f4eb6baaff12d7ebe11dde6348.svg
paper_authors.fullname: Vardaan Pahuja
paper_authors.type: user
paper_authors.user: vardaan123
paper_authors._id: 67b3f1f1f5bd60d66133e1f4
paper_authors.name: Yadong Lu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-02-18T02:35:29.988Z
paper_authors._id: 664bbd75a6bd1b3d2ac7fc34
paper_authors.avatarUrl: /avatars/127bf5d611b46ef95a1859a8cf21a160.svg
paper_authors.fullname: Yadong Lu
paper_authors.type: user
paper_authors.user: adamlu1
paper_authors._id: 67b3f1f1f5bd60d66133e1f5
paper_authors.name: Corby Rosset
paper_authors._id: 67b3f1f1f5bd60d66133e1f6
paper_authors.name: Boyu Gou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T17:33:00.709Z
paper_authors._id: 6500870f1e14749e84f8f887
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6500870f1e14749e84f8f887/wfvx4BZvh2OyW-vpq5jEy.jpeg
paper_authors.fullname: Boyu Gou
paper_authors.type: user
paper_authors.user: BoyuNLP
paper_authors._id: 67b3f1f1f5bd60d66133e1f7
paper_authors.name: Arindam Mitra
paper_authors._id: 67b3f1f1f5bd60d66133e1f8
paper_authors.name: Spencer Whitehead
paper_authors._id: 67b3f1f1f5bd60d66133e1f9
paper_authors.name: Yu Su
paper_authors._id: 67b3f1f1f5bd60d66133e1fa
paper_authors.name: Ahmed Awadallah
paper_title: Explorer: Scaling Exploration-driven Web Trajectory Synthesis for
    Multimodal Web Agents
paper_summary: Recent success in large multimodal models (LMMs) has sparked promising
  applications of agents capable of autonomously completing complex web tasks.
  While open-source LMM agents have made significant advances in offline
  evaluation benchmarks, their performance still falls substantially short of
  human-level capabilities in more realistic online settings. A key bottleneck is
  the lack of diverse and large-scale trajectory-level datasets across various
  domains, which are expensive to collect. In this paper, we address this
  challenge by developing a scalable recipe to synthesize the largest and most
  diverse trajectory-level dataset to date, containing over 94K successful
  multimodal web trajectories, spanning 49K unique URLs, 720K screenshots, and
  33M web elements. In particular, we leverage extensive web exploration and
  refinement to obtain diverse task intents. The average cost is 28 cents per
  successful trajectory, making it affordable to a wide range of users in the
  community. Leveraging this dataset, we train Explorer, a multimodal web agent,
  and demonstrate strong performance on both offline and online web agent
  benchmarks such as Mind2Web-Live, Multimodal-Mind2Web, and MiniWob++.
  Additionally, our experiments highlight data scaling as a key driver for
  improving web agent capabilities. We hope this study makes state-of-the-art
  LMM-based agent research at a larger scale more accessible.
paper_ai_summary: A scalable dataset generates diverse multimodal web trajectories to enhance performance of LMM-based web agents.
paper_ai_keywords: multimodal models LMMs trajectory-level datasets web exploration multimodal web agent web agent benchmarks Mind2Web-Live Multimodal-Mind2Web MiniWob++