Agent Workflow Memory
Despite the potential of language model-based agents to solve real-world
tasks such as web navigation, current methods still struggle with long-horizon
tasks with complex action trajectories. In contrast, humans can flexibly solve
complex tasks by learning reusable task workflows from past experiences and
using them to guide future actions. To build agents that can similarly benefit
from this process, we introduce Agent Workflow Memory (AWM), a method for
inducing commonly reused routines, i.e., workflows, and selectively providing
workflows to the agent to guide subsequent generations. AWM flexibly applies to
both offline and online scenarios, where agents induce workflows from training
examples beforehand or from test queries on the fly. We experiment on two major
web navigation benchmarks -- Mind2Web and WebArena -- that collectively cover
1000+ tasks from 200+ domains across travel, shopping, and social media, among
others. AWM substantially improves the baseline results by 24.6% and 51.1%
relative success rate on Mind2Web and WebArena while reducing the number of
steps taken to solve WebArena tasks successfully. Furthermore, online AWM
robustly generalizes in cross-task, website, and domain evaluations, surpassing
baselines from 8.9 to 14.0 absolute points as train-test task distribution gaps
widen.
66e254c524df851611e7f5a6
Zora Zhiruo Wang
claimed_verified
2025-01-07T16:14:37.227Z
6356c77dd18276171735985f
https://cdn-avatars.huggingface.co/v1/production/uploads/6356c77dd18276171735985f/HYsRr1G13xgwVUoou7nFQ.jpeg
Zora Zhiruo Wang
user
zorazrw
66e254c524df851611e7f5a7
Jiayuan Mao
admin_assigned
2024-09-12T07:19:15.447Z
62f168243e0991a8ab14311d
/avatars/a5c118a5bd7ce4a49af752068d60ccea.svg
Mao
user
Jiayuan
66e254c524df851611e7f5a8
Daniel Fried
66e254c524df851611e7f5a9
Graham Neubig
admin_assigned
2024-09-12T07:19:25.474Z
60de14638bedd2315529d43f
https://cdn-avatars.huggingface.co/v1/production/uploads/1625166923504-noauth.png
Graham Neubig
user
gneubig
Agent Workflow Memory
Despite the potential of language model-based agents to solve real-world
tasks such as web navigation, current methods still struggle with long-horizon
tasks with complex action trajectories. In contrast, humans can flexibly solve
complex tasks by learning reusable task workflows from past experiences and
using them to guide future actions. To build agents that can similarly benefit
from this process, we introduce Agent Workflow Memory (AWM), a method for
inducing commonly reused routines, i.e., workflows, and selectively providing
workflows to the agent to guide subsequent generations. AWM flexibly applies to
both offline and online scenarios, where agents induce workflows from training
examples beforehand or from test queries on the fly. We experiment on two major
web navigation benchmarks -- Mind2Web and WebArena -- that collectively cover
1000+ tasks from 200+ domains across travel, shopping, and social media, among
others. AWM substantially improves the baseline results by 24.6% and 51.1%
relative success rate on Mind2Web and WebArena while reducing the number of
steps taken to solve WebArena tasks successfully. Furthermore, online AWM
robustly generalizes in cross-task, website, and domain evaluations, surpassing
baselines from 8.9 to 14.0 absolute points as train-test task distribution gaps
widen.
Agent Workflow Memory (AWM) enhances language model-based agents' performance on complex tasks by inducing and using reusable task workflows, resulting in improved success rates and reduced steps compared to baselines.
Agent Workflow Memory AWM long-horizon tasks complex action trajectories task workflows reusable routines offline scenarios online scenarios Mind2Web WebArena relative success rate cross-task generalization website generalization domain evaluation