Towards A Unified Agent with Foundation Models
Language Models and Vision Language Models have recently demonstrated
unprecedented capabilities in terms of understanding human intentions,
reasoning, scene understanding, and planning-like behaviour, in text form,
among many others. In this work, we investigate how to embed and leverage such
abilities in Reinforcement Learning (RL) agents. We design a framework that
uses language as the core reasoning tool, exploring how this enables an agent
to tackle a series of fundamental RL challenges, such as efficient exploration,
reusing experience data, scheduling skills, and learning from observations,
which traditionally require separate, vertically designed algorithms. We test
our method on a sparse-reward simulated robotic manipulation environment, where
a robot needs to stack a set of objects. We demonstrate substantial performance
improvements over baselines in exploration efficiency and ability to reuse data
from offline datasets, and illustrate how to reuse learned skills to solve
novel tasks or imitate videos of human experts.
64b89aca7ac999b4a14bab80
Norman Di Palo
claimed_verified
2023-07-21T07:36:24.517Z
62dac377f014388f908974f4
/avatars/39bf0ca206441575a45f577060cdd8bc.svg
Norman Di Palo
user
normandipalo
64b89aca7ac999b4a14bab81
Arunkumar Byravan
64b89aca7ac999b4a14bab82
Leonard Hasenclever
64b89aca7ac999b4a14bab83
Markus Wulfmeier
64b89aca7ac999b4a14bab84
Nicolas Heess
64b89aca7ac999b4a14bab85
Martin Riedmiller
Towards A Unified Agent with Foundation Models
Language Models and Vision Language Models have recently demonstrated
unprecedented capabilities in terms of understanding human intentions,
reasoning, scene understanding, and planning-like behaviour, in text form,
among many others. In this work, we investigate how to embed and leverage such
abilities in Reinforcement Learning (RL) agents. We design a framework that
uses language as the core reasoning tool, exploring how this enables an agent
to tackle a series of fundamental RL challenges, such as efficient exploration,
reusing experience data, scheduling skills, and learning from observations,
which traditionally require separate, vertically designed algorithms. We test
our method on a sparse-reward simulated robotic manipulation environment, where
a robot needs to stack a set of objects. We demonstrate substantial performance
improvements over baselines in exploration efficiency and ability to reuse data
from offline datasets, and illustrate how to reuse learned skills to solve
novel tasks or imitate videos of human experts.
A framework using language as a reasoning tool in RL agents improves exploration efficiency and data reuse, and enhances the acquisition of skills for novel tasks.
language models vision language models reinforcement learning (RL) efficient exploration experience data reuse scheduling skills learning from observations sparse-reward environment robotic manipulation offline datasets skill reuse imitating human experts