title: Web Agents with World Models: Learning and Leveraging Environment
    Dynamics in Web Navigation
summary: Large language models (LLMs) have recently gained much attention in building
  autonomous agents. However, the performance of current LLM-based web agents in
  long-horizon tasks is far from optimal, often yielding errors such as
  repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid
  such an irreversible mistake, as we have an awareness of the potential outcomes
  (e.g., losing money) of our actions, also known as the "world model". Motivated
  by this, our study first starts with preliminary analyses, confirming the
  absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet,
  etc.). Then, we present a World-model-augmented (WMA) web agent, which
  simulates the outcomes of its actions for better decision-making. To overcome
  the challenges in training LLMs as world models predicting next observations,
  such as repeated elements across observations and long HTML inputs, we propose
  a transition-focused observation abstraction, where the prediction objectives
  are free-form natural language descriptions exclusively highlighting important
  state differences between time steps. Experiments on WebArena and Mind2Web show
  that our world models improve agents' policy selection without training and
  demonstrate our agents' cost- and time-efficiency compared to recent
  tree-search-based agents.
paper_authors._id: 6715f0338cb36b73b1dbf48d
paper_authors.name: Hyungjoo Chae
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T10:26:53.304Z
paper_authors._id: 64c8f4cec547ed5243ebd0a8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c8f4cec547ed5243ebd0a8/MiOH5YbMg8Gh9KYlQsLmX.jpeg
paper_authors.fullname: Hyungjoo Chae
paper_authors.type: user
paper_authors.user: hyungjoochae
paper_authors._id: 6715f0338cb36b73b1dbf48e
paper_authors.name: Namyoung Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T11:45:10.010Z
paper_authors._id: 6631e2b9fb55832ca41d78b1
paper_authors.avatarUrl: /avatars/6115ac6dfb3cc6c8e6e94729f3d5a8c5.svg
paper_authors.fullname: Namyoung Kim
paper_authors.type: user
paper_authors.user: kimnamssya
paper_authors._id: 6715f0338cb36b73b1dbf48f
paper_authors.name: Kai Tzu-iunn Ong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T11:43:52.128Z
paper_authors._id: 640ec2fd2f9c7b364d182735
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640ec2fd2f9c7b364d182735/G5WxQXYEOaVL39rAoOI3Y.jpeg
paper_authors.fullname: kai tzu-iunn ong
paper_authors.type: user
paper_authors.user: ktio
paper_authors._id: 6715f0338cb36b73b1dbf490
paper_authors.name: Minju Gwak
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-23T07:35:16.372Z
paper_authors._id: 6635a672b0a5f86a2aeacd59
paper_authors.avatarUrl: /avatars/371529d2d5a858d1c26858494ca9722e.svg
paper_authors.fullname: Minju Gwak
paper_authors.type: user
paper_authors.user: talzoomanzoo
paper_authors._id: 6715f0338cb36b73b1dbf491
paper_authors.name: Gwanwoo Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T11:43:39.177Z
paper_authors._id: 668d512d3d0bc7c4758a73cc
paper_authors.avatarUrl: /avatars/054e4ed3c06f0464fb06b88fa7cec7f3.svg
paper_authors.fullname: Gwanwoo Song
paper_authors.type: user
paper_authors.user: Gwanwoo
paper_authors._id: 6715f0338cb36b73b1dbf492
paper_authors.name: Jihoon Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T11:44:42.881Z
paper_authors._id: 66429ceee4a7476619a5ab64
paper_authors.avatarUrl: /avatars/43cb4f087f4f9dc972196047fdb339f4.svg
paper_authors.fullname: Jihoon Kim
paper_authors.type: user
paper_authors.user: jihoonkim25
paper_authors._id: 6715f0338cb36b73b1dbf493
paper_authors.name: Sunghwan Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-23T15:06:54.954Z
paper_authors._id: 646a0897c37ca1e12308b026
paper_authors.avatarUrl: /avatars/6d720a9e366db9bec15c8c10878c0c75.svg
paper_authors.fullname: Sunghwan Kim
paper_authors.type: user
paper_authors.user: KimSHine
paper_authors._id: 6715f0338cb36b73b1dbf494
paper_authors.name: Dongha Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T12:14:10.468Z
paper_authors._id: 61d4bb8a34cff1c8e64b6a2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61d4bb8a34cff1c8e64b6a2a/eEXKbsXB9VK1sR8QqNPi7.png
paper_authors.fullname: Dongha Lee
paper_authors.type: user
paper_authors.user: donalee
paper_authors._id: 6715f0338cb36b73b1dbf495
paper_authors.name: Jinyoung Yeo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T11:43:28.091Z
paper_authors._id: 6419ad74060a651c415eec62
paper_authors.avatarUrl: /avatars/e7b060d0a45e3af1c7439948dbd60e34.svg
paper_authors.fullname: Jinyoung Yeom
paper_authors.type: user
paper_authors.user: geen02
paper_title: Web Agents with World Models: Learning and Leveraging Environment
    Dynamics in Web Navigation
paper_summary: Large language models (LLMs) have recently gained much attention in building
  autonomous agents. However, the performance of current LLM-based web agents in
  long-horizon tasks is far from optimal, often yielding errors such as
  repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid
  such an irreversible mistake, as we have an awareness of the potential outcomes
  (e.g., losing money) of our actions, also known as the "world model". Motivated
  by this, our study first starts with preliminary analyses, confirming the
  absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet,
  etc.). Then, we present a World-model-augmented (WMA) web agent, which
  simulates the outcomes of its actions for better decision-making. To overcome
  the challenges in training LLMs as world models predicting next observations,
  such as repeated elements across observations and long HTML inputs, we propose
  a transition-focused observation abstraction, where the prediction objectives
  are free-form natural language descriptions exclusively highlighting important
  state differences between time steps. Experiments on WebArena and Mind2Web show
  that our world models improve agents' policy selection without training and
  demonstrate our agents' cost- and time-efficiency compared to recent
  tree-search-based agents.
paper_ai_keywords: Large language models (LLMs) web agents long-horizon tasks world model GPT-4o Claude-3.5-Sonnet World-model-augmented (WMA) web agent transition-focused observation abstraction WebArena Mind2Web policy selection tree-search-based agents