Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL
<|reference_start|>Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL: Although there is a growing demand for cooking behaviours as one of the expected tasks for robots, a series of cooking behaviours based on new recipe descriptions by robots in the real world has not yet been realised. In this study, we propose a robot system that integrates real-world executable robot cooking behaviour planning using the Large Language Model (LLM) and classical planning of PDDL descriptions, and food ingredient state recognition learning from a small number of data using the Vision-Language model (VLM). We succeeded in experiments in which PR2, a dual-armed wheeled robot, performed cooking from arranged new recipes in a real-world environment, and confirmed the effectiveness of the proposed system.<|reference_end|>
@article{kanazawa2024real-world,
    title={Real-World Cooking Robot System from Recipes Based on Food State
  Recognition Using Foundation Models and PDDL},
    author={Naoaki Kanazawa, Kento Kawaharazuka, Yoshiki Obinata, Kei Okada,
  Masayuki Inaba},
    journal={arXiv preprint arXiv:2410.02874},
    year={2024},
    doi={10.1080/01691864.2024.2407136},
    archivePrefix={arXiv},
    eprint={2410.02874},
    primaryClass={cs.RO cs.AI}
}