ReLEP: A Novel Framework for Real-world Long-horizon Embodied Planning
<|reference_start|>ReLEP: A Novel Framework for Real-world Long-horizon Embodied Planning: Real-world long-horizon embodied planning underpins embodied AI. To accomplish long-horizon tasks, agents need to decompose abstract instructions into detailed steps. Prior works mostly rely on GPT-4V for task decomposition into predefined actions, which limits task diversity due to GPT-4V's finite understanding of larger skillsets. Therefore, we present ReLEP, a groundbreaking framework for Real world Long-horizon Embodied Planning, which can accomplish a wide range of daily tasks. At its core lies a fine-tuned large vision language model that formulates plans as sequences of skill functions according to input instruction and scene image. These functions are selected from a carefully designed skill library. ReLEP is also equipped with a Memory module for plan and status recall, and a Robot Configuration module for versatility across robot types. In addition, we propose a semi-automatic data generation pipeline to tackle dataset scarcity. Real-world off-line experiments across eight daily embodied tasks demonstrate that ReLEP is able to accomplish long-horizon embodied tasks and outperforms other state-of-the-art baseline methods.<|reference_end|>
@article{liu2024relep:,
    title={ReLEP: A Novel Framework for Real-world Long-horizon Embodied Planning},
    author={Siyuan Liu, Jiawei Du, Sicheng Xiang, Zibo Wang and Dingsheng Luo},
    journal={arXiv preprint arXiv:2409.15658},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.15658},
    primaryClass={cs.RO cs.AI}
}