Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation
<|reference_start|>Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation: Large Language Models (LLMs) have been recently used in robot applications for grounding LLM common-sense reasoning with the robot's perception and physical abilities. In humanoid robots, memory also plays a critical role in fostering real-world embodiment and facilitating long-term interactive capabilities, especially in multi-task setups where the robot must remember previous task states, environment states, and executed actions. In this paper, we address incorporating memory processes with LLMs for generating cross-task robot actions, while the robot effectively switches between tasks. Our proposed dual-layered architecture features two LLMs, utilizing their complementary skills of reasoning and following instructions, combined with a memory model inspired by human cognition. Our results show a significant improvement in performance over a baseline of five robotic tasks, demonstrating the potential of integrating memory with LLMs for combining the robot's action and perception for adaptive task execution.<|reference_end|>
@article{ali2024robots,
    title={Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for
  Enhanced Cross-Task Robot Action Generation},
    author={Hassan Ali, Philipp Allgeuer, Carlo Mazzola, Giulia Belgiovine, Burak
  Can Kaplan, Luk'av{s} Gajdov{s}ech, Stefan Wermter},
    journal={arXiv preprint arXiv:2407.13505},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13505},
    primaryClass={cs.RO cs.AI}
}