On the Multi-turn Instruction Following for Conversational Web Agents
<|reference_start|>On the Multi-turn Instruction Following for Conversational Web Agents: Web agents powered by Large Language Models (LLMs) have demonstrated remarkable abilities in planning and executing multi-step interactions within complex web-based environments, fulfilling a wide range of web navigation tasks. Despite these advancements, the potential for LLM-powered agents to effectively engage with sequential user instructions in real-world scenarios has not been fully explored. In this work, we introduce a new task of Conversational Web Navigation, which necessitates sophisticated interactions that span multiple turns with both the users and the environment, supported by a specially developed dataset named Multi-Turn Mind2Web (MT-Mind2Web). To tackle the limited context length of LLMs and the context-dependency issue of the conversational tasks, we further propose a novel framework, named self-reflective memory-augmented planning (Self-MAP), which employs memory utilization and self-reflection techniques. Extensive experiments are conducted to benchmark the MT-Mind2Web dataset, and validate the effectiveness of the proposed method.<|reference_end|>
@article{deng2024on,
    title={On the Multi-turn Instruction Following for Conversational Web Agents},
    author={Yang Deng, Xuan Zhang, Wenxuan Zhang, Yifei Yuan, See-Kiong Ng,
  Tat-Seng Chua},
    journal={arXiv preprint arXiv:2402.15057},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.15057},
    primaryClass={cs.CL cs.AI}
}