LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning
<|reference_start|>LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning: Conventional Task and Motion Planning (TAMP) approaches rely on manually crafted interfaces connecting symbolic task planning with continuous motion generation. These domain-specific and labor-intensive modules are limited in addressing emerging tasks in real-world settings. Here, we present LLM^3, a novel Large Language Model (LLM)-based TAMP framework featuring a domain-independent interface. Specifically, we leverage the powerful reasoning and planning capabilities of pre-trained LLMs to propose symbolic action sequences and select continuous action parameters for motion planning. Crucially, LLM^3 incorporates motion planning feedback through prompting, allowing the LLM to iteratively refine its proposals by reasoning about motion failure. Consequently, LLM^3 interfaces between task planning and motion planning, alleviating the intricate design process of handling domain-specific messages between them. Through a series of simulations in a box-packing domain, we quantitatively demonstrate the effectiveness of LLM^3 in solving TAMP problems and the efficiency in selecting action parameters. Ablation studies underscore the significant contribution of motion failure reasoning to the success of LLM^3. Furthermore, we conduct qualitative experiments on a physical manipulator, demonstrating the practical applicability of our approach in real-world settings.<|reference_end|>
@article{wang2024llm3:large,
    title={LLM3:Large Language Model-based Task and Motion Planning with Motion
  Failure Reasoning},
    author={Shu Wang, Muzhi Han, Ziyuan Jiao, Zeyu Zhang, Ying Nian Wu, Song-Chun
  Zhu, Hangxin Liu},
    journal={arXiv preprint arXiv:2403.11552},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.11552},
    primaryClass={cs.RO cs.AI}
}