OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World
  Exploration, Feedback and Optimization
The rapid development of large language and multimodal models has sparked
significant interest in using proprietary models, such as GPT-4o, to develop
autonomous agents capable of handling real-world scenarios like web navigation.
Although recent open-source efforts have tried to equip agents with the ability
to explore environments and continuously improve over time, they are building
text-only agents in synthetic environments where the reward signals are clearly
defined. Such agents struggle to generalize to realistic settings that require
multimodal perception abilities and lack ground-truth signals. In this paper,
we introduce an open-source framework designed to facilitate the development of
multimodal web agent that can autonomously conduct real-world exploration and
improve itself. We first train the base model with imitation learning to gain
the basic abilities. We then let the agent explore the open web and collect
feedback on its trajectories. After that, it further improves its policy by
learning from well-performing trajectories judged by another general-purpose
model. This exploration-feedback-optimization cycle can continue for several
iterations. Experimental results show that our web agent successfully improves
itself after each iteration, demonstrating strong performance across multiple
test sets.
6720ac237fb8a490619de385
Hongliang He
6720ac237fb8a490619de386
Wenlin Yao
6720ac237fb8a490619de387
Kaixin Ma
6720ac237fb8a490619de388
Wenhao Yu
6720ac237fb8a490619de389
Hongming Zhang
6720ac237fb8a490619de38a
Tianqing Fang
6720ac237fb8a490619de38b
Zhenzhong Lan
6720ac237fb8a490619de38c
Dong Yu
OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World
  Exploration, Feedback and Optimization
The rapid development of large language and multimodal models has sparked
significant interest in using proprietary models, such as GPT-4o, to develop
autonomous agents capable of handling real-world scenarios like web navigation.
Although recent open-source efforts have tried to equip agents with the ability
to explore environments and continuously improve over time, they are building
text-only agents in synthetic environments where the reward signals are clearly
defined. Such agents struggle to generalize to realistic settings that require
multimodal perception abilities and lack ground-truth signals. In this paper,
we introduce an open-source framework designed to facilitate the development of
multimodal web agent that can autonomously conduct real-world exploration and
improve itself. We first train the base model with imitation learning to gain
the basic abilities. We then let the agent explore the open web and collect
feedback on its trajectories. After that, it further improves its policy by
learning from well-performing trajectories judged by another general-purpose
model. This exploration-feedback-optimization cycle can continue for several
iterations. Experimental results show that our web agent successfully improves
itself after each iteration, demonstrating strong performance across multiple
test sets.
An open-source framework enables the development of a multimodal web agent that autonomously explores and improves through an imitation learning and exploration-feedback-optimization cycle.
large language models multimodal models GPT-4o imitation learning exploration open web trajectories policy improvement