title: SteP: Stacked LLM Policies for Web Actions
summary: Performing tasks on the web presents fundamental challenges to large language
  models (LLMs), including combinatorially large open-world tasks and variations
  across web interfaces. Simply specifying a large prompt to handle all possible
  behaviors and states is extremely complex, and results in behavior leaks
  between unrelated behaviors. Decomposition to distinct policies can address
  this challenge, but requires carefully handing off control between policies. We
  propose Stacked LLM Policies for Web Actions (SteP), an approach to dynamically
  compose policies to solve a diverse set of web tasks. SteP defines a Markov
  Decision Process where the state is a stack of policies representing the
  control state, i.e., the chain of policy calls. Unlike traditional methods that
  are restricted to static hierarchies, SteP enables dynamic control that adapts
  to the complexity of the task. We evaluate SteP against multiple baselines and
  web environments including WebArena, MiniWoB++, and a CRM. On WebArena, SteP
  improves (14.9\% to 33.5\%) over SOTA that use GPT-4 policies, while on
  MiniWob++, SteP is competitive with prior works while using significantly less
  data. Our code and data are available at
  https://asappresearch.github.io/webagents-step.
paper_authors._id: 651f6808e17ef6a94e063719
paper_authors.name: Paloma Sodhi
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-10-06T01:51:08.453Z
paper_authors._id: 64e0c88e4d740812bb32b4fc
paper_authors.avatarUrl: /avatars/67abdcebc68c0dbc88c71e1a26349e22.svg
paper_authors.fullname: Paloma Sodhi
paper_authors.type: user
paper_authors.user: psodhi-asapp
paper_authors._id: 651f6808e17ef6a94e06371a
paper_authors.name: S. R. K. Branavan
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-01-31T19:30:07.860Z
paper_authors._id: 65ba9f9b7677ff48f59e8c3c
paper_authors.avatarUrl: /avatars/22b8e6a9a341ab83b6977d80a9822d34.svg
paper_authors.fullname: S.R.K. Branavan
paper_authors.type: user
paper_authors.user: branavan-asapp
paper_authors._id: 651f6808e17ef6a94e06371b
paper_authors.name: Ryan McDonald
paper_title: SteP: Stacked LLM Policies for Web Actions
paper_summary: Performing tasks on the web presents fundamental challenges to large language
  models (LLMs), including combinatorially large open-world tasks and variations
  across web interfaces. Simply specifying a large prompt to handle all possible
  behaviors and states is extremely complex, and results in behavior leaks
  between unrelated behaviors. Decomposition to distinct policies can address
  this challenge, but requires carefully handing off control between policies. We
  propose Stacked LLM Policies for Web Actions (SteP), an approach to dynamically
  compose policies to solve a diverse set of web tasks. SteP defines a Markov
  Decision Process where the state is a stack of policies representing the
  control state, i.e., the chain of policy calls. Unlike traditional methods that
  are restricted to static hierarchies, SteP enables dynamic control that adapts
  to the complexity of the task. We evaluate SteP against multiple baselines and
  web environments including WebArena, MiniWoB++, and a CRM. On WebArena, SteP
  improves (14.9\% to 33.5\%) over SOTA that use GPT-4 policies, while on
  MiniWob++, SteP is competitive with prior works while using significantly less
  data. Our code and data are available at
  https://asappresearch.github.io/webagents-step.
paper_ai_keywords: Stacked LLM Policies Markov Decision Process WebArena MiniWoB++ CRM SOTA (State of the Art)