WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents
<|reference_start|>WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents: In the realm of web agent research, achieving both generalization and accuracy remains a challenging problem. Due to high variance in website structure, existing approaches often fail. Moreover, existing fine-tuning and in-context learning techniques fail to generalize across multiple websites. We introduce Wilbur, an approach that uses a differentiable ranking model and a novel instruction synthesis technique to optimally populate a black-box large language model's prompt with task demonstrations from previous runs. To maximize end-to-end success rates, we also propose an intelligent backtracking mechanism that learns and recovers from its mistakes. Finally, we show that our ranking model can be trained on data from a generative auto-curriculum which samples representative goals from an LLM, runs the agent, and automatically evaluates it, with no manual annotation. Wilbur achieves state-of-the-art results on the WebVoyager benchmark, beating text-only models by 8% overall, and up to 36% on certain websites. On the same benchmark, Wilbur is within 5% of a strong multi-modal model despite only receiving textual inputs, and further analysis reveals a substantial number of failures are due to engineering challenges of operating the web.<|reference_end|>
@article{lutz2024wilbur:,
    title={WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents},
    author={Michael Lutz, Arth Bohra, Manvel Saroyan, Artem Harutyunyan, Giovanni
  Campagna},
    journal={arXiv preprint arXiv:2404.05902},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.05902},
    primaryClass={cs.CL cs.AI}
}