Web Agents with World Models: Learning and Leveraging Environment
  Dynamics in Web Navigation
Large language models (LLMs) have recently gained much attention in building
autonomous agents. However, the performance of current LLM-based web agents in
long-horizon tasks is far from optimal, often yielding errors such as
repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid
such an irreversible mistake, as we have an awareness of the potential outcomes
(e.g., losing money) of our actions, also known as the "world model". Motivated
by this, our study first starts with preliminary analyses, confirming the
absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet,
etc.). Then, we present a World-model-augmented (WMA) web agent, which
simulates the outcomes of its actions for better decision-making. To overcome
the challenges in training LLMs as world models predicting next observations,
such as repeated elements across observations and long HTML inputs, we propose
a transition-focused observation abstraction, where the prediction objectives
are free-form natural language descriptions exclusively highlighting important
state differences between time steps. Experiments on WebArena and Mind2Web show
that our world models improve agents' policy selection without training and
demonstrate our agents' cost- and time-efficiency compared to recent
tree-search-based agents.
6715f0338cb36b73b1dbf48d
Hyungjoo Chae
admin_assigned
2024-10-21T10:26:53.304Z
64c8f4cec547ed5243ebd0a8
https://cdn-avatars.huggingface.co/v1/production/uploads/64c8f4cec547ed5243ebd0a8/MiOH5YbMg8Gh9KYlQsLmX.jpeg
Hyungjoo Chae
user
hyungjoochae
6715f0338cb36b73b1dbf48e
Namyoung Kim
admin_assigned
2024-10-21T11:45:10.010Z
6631e2b9fb55832ca41d78b1
/avatars/6115ac6dfb3cc6c8e6e94729f3d5a8c5.svg
Namyoung Kim
user
kimnamssya
6715f0338cb36b73b1dbf48f
Kai Tzu-iunn Ong
admin_assigned
2024-10-21T11:43:52.128Z
640ec2fd2f9c7b364d182735
https://cdn-avatars.huggingface.co/v1/production/uploads/640ec2fd2f9c7b364d182735/G5WxQXYEOaVL39rAoOI3Y.jpeg
kai tzu-iunn ong
user
ktio
6715f0338cb36b73b1dbf490
Minju Gwak
claimed_verified
2024-10-23T07:35:16.372Z
6635a672b0a5f86a2aeacd59
/avatars/371529d2d5a858d1c26858494ca9722e.svg
Minju Gwak
user
talzoomanzoo
6715f0338cb36b73b1dbf491
Gwanwoo Song
admin_assigned
2024-10-21T11:43:39.177Z
668d512d3d0bc7c4758a73cc
/avatars/054e4ed3c06f0464fb06b88fa7cec7f3.svg
Gwanwoo Song
user
Gwanwoo
6715f0338cb36b73b1dbf492
Jihoon Kim
admin_assigned
2024-10-21T11:44:42.881Z
66429ceee4a7476619a5ab64
/avatars/43cb4f087f4f9dc972196047fdb339f4.svg
Jihoon Kim
user
jihoonkim25
6715f0338cb36b73b1dbf493
Sunghwan Kim
claimed_verified
2024-10-23T15:06:54.954Z
646a0897c37ca1e12308b026
/avatars/6d720a9e366db9bec15c8c10878c0c75.svg
Sunghwan Kim
user
KimSHine
6715f0338cb36b73b1dbf494
Dongha Lee
claimed_verified
2024-10-21T12:14:10.468Z
61d4bb8a34cff1c8e64b6a2a
https://cdn-avatars.huggingface.co/v1/production/uploads/61d4bb8a34cff1c8e64b6a2a/eEXKbsXB9VK1sR8QqNPi7.png
Dongha Lee
user
donalee
6715f0338cb36b73b1dbf495
Jinyoung Yeo
admin_assigned
2024-10-21T11:43:28.091Z
6419ad74060a651c415eec62
/avatars/e7b060d0a45e3af1c7439948dbd60e34.svg
Jinyoung Yeom
user
geen02
Web Agents with World Models: Learning and Leveraging Environment
  Dynamics in Web Navigation
Large language models (LLMs) have recently gained much attention in building
autonomous agents. However, the performance of current LLM-based web agents in
long-horizon tasks is far from optimal, often yielding errors such as
repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid
such an irreversible mistake, as we have an awareness of the potential outcomes
(e.g., losing money) of our actions, also known as the "world model". Motivated
by this, our study first starts with preliminary analyses, confirming the
absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet,
etc.). Then, we present a World-model-augmented (WMA) web agent, which
simulates the outcomes of its actions for better decision-making. To overcome
the challenges in training LLMs as world models predicting next observations,
such as repeated elements across observations and long HTML inputs, we propose
a transition-focused observation abstraction, where the prediction objectives
are free-form natural language descriptions exclusively highlighting important
state differences between time steps. Experiments on WebArena and Mind2Web show
that our world models improve agents' policy selection without training and
demonstrate our agents' cost- and time-efficiency compared to recent
tree-search-based agents.
A World-model-augmented web agent improves decision-making by simulating action outcomes, overcoming training challenges with free-form natural language descriptions of state differences.
world model LLM-based web agents long-horizon tasks web agents GPT-4o Claude-3.5-Sonnet World-model-augmented (WMA) web agent transition-focused observation abstraction WebArena Mind2Web cost-efficiency time-efficiency tree-search-based agents