title: OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World
    Exploration, Feedback and Optimization
summary: The rapid development of large language and multimodal models has sparked
  significant interest in using proprietary models, such as GPT-4o, to develop
  autonomous agents capable of handling real-world scenarios like web navigation.
  Although recent open-source efforts have tried to equip agents with the ability
  to explore environments and continuously improve over time, they are building
  text-only agents in synthetic environments where the reward signals are clearly
  defined. Such agents struggle to generalize to realistic settings that require
  multimodal perception abilities and lack ground-truth signals. In this paper,
  we introduce an open-source framework designed to facilitate the development of
  multimodal web agent that can autonomously conduct real-world exploration and
  improve itself. We first train the base model with imitation learning to gain
  the basic abilities. We then let the agent explore the open web and collect
  feedback on its trajectories. After that, it further improves its policy by
  learning from well-performing trajectories judged by another general-purpose
  model. This exploration-feedback-optimization cycle can continue for several
  iterations. Experimental results show that our web agent successfully improves
  itself after each iteration, demonstrating strong performance across multiple
  test sets.
paper_authors._id: 6720ac237fb8a490619de385
paper_authors.name: Hongliang He
paper_authors._id: 6720ac237fb8a490619de386
paper_authors.name: Wenlin Yao
paper_authors._id: 6720ac237fb8a490619de387
paper_authors.name: Kaixin Ma
paper_authors._id: 6720ac237fb8a490619de388
paper_authors.name: Wenhao Yu
paper_authors._id: 6720ac237fb8a490619de389
paper_authors.name: Hongming Zhang
paper_authors._id: 6720ac237fb8a490619de38a
paper_authors.name: Tianqing Fang
paper_authors._id: 6720ac237fb8a490619de38b
paper_authors.name: Zhenzhong Lan
paper_authors._id: 6720ac237fb8a490619de38c
paper_authors.name: Dong Yu
paper_title: OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World
    Exploration, Feedback and Optimization
paper_summary: The rapid development of large language and multimodal models has sparked
  significant interest in using proprietary models, such as GPT-4o, to develop
  autonomous agents capable of handling real-world scenarios like web navigation.
  Although recent open-source efforts have tried to equip agents with the ability
  to explore environments and continuously improve over time, they are building
  text-only agents in synthetic environments where the reward signals are clearly
  defined. Such agents struggle to generalize to realistic settings that require
  multimodal perception abilities and lack ground-truth signals. In this paper,
  we introduce an open-source framework designed to facilitate the development of
  multimodal web agent that can autonomously conduct real-world exploration and
  improve itself. We first train the base model with imitation learning to gain
  the basic abilities. We then let the agent explore the open web and collect
  feedback on its trajectories. After that, it further improves its policy by
  learning from well-performing trajectories judged by another general-purpose
  model. This exploration-feedback-optimization cycle can continue for several
  iterations. Experimental results show that our web agent successfully improves
  itself after each iteration, demonstrating strong performance across multiple
  test sets.
paper_ai_keywords: imitation learning multimodal real-world exploration feedback well-performing trajectories policy learning