title: Towards A Unified Agent with Foundation Models
summary: Language Models and Vision Language Models have recently demonstrated
  unprecedented capabilities in terms of understanding human intentions,
  reasoning, scene understanding, and planning-like behaviour, in text form,
  among many others. In this work, we investigate how to embed and leverage such
  abilities in Reinforcement Learning (RL) agents. We design a framework that
  uses language as the core reasoning tool, exploring how this enables an agent
  to tackle a series of fundamental RL challenges, such as efficient exploration,
  reusing experience data, scheduling skills, and learning from observations,
  which traditionally require separate, vertically designed algorithms. We test
  our method on a sparse-reward simulated robotic manipulation environment, where
  a robot needs to stack a set of objects. We demonstrate substantial performance
  improvements over baselines in exploration efficiency and ability to reuse data
  from offline datasets, and illustrate how to reuse learned skills to solve
  novel tasks or imitate videos of human experts.
paper_authors._id: 64b89aca7ac999b4a14bab80
paper_authors.name: Norman Di Palo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-21T07:36:24.517Z
paper_authors._id: 62dac377f014388f908974f4
paper_authors.avatarUrl: /avatars/39bf0ca206441575a45f577060cdd8bc.svg
paper_authors.fullname: Norman Di Palo
paper_authors.type: user
paper_authors.user: normandipalo
paper_authors._id: 64b89aca7ac999b4a14bab81
paper_authors.name: Arunkumar Byravan
paper_authors._id: 64b89aca7ac999b4a14bab82
paper_authors.name: Leonard Hasenclever
paper_authors._id: 64b89aca7ac999b4a14bab83
paper_authors.name: Markus Wulfmeier
paper_authors._id: 64b89aca7ac999b4a14bab84
paper_authors.name: Nicolas Heess
paper_authors._id: 64b89aca7ac999b4a14bab85
paper_authors.name: Martin Riedmiller
paper_title: Towards A Unified Agent with Foundation Models
paper_summary: Language Models and Vision Language Models have recently demonstrated
  unprecedented capabilities in terms of understanding human intentions,
  reasoning, scene understanding, and planning-like behaviour, in text form,
  among many others. In this work, we investigate how to embed and leverage such
  abilities in Reinforcement Learning (RL) agents. We design a framework that
  uses language as the core reasoning tool, exploring how this enables an agent
  to tackle a series of fundamental RL challenges, such as efficient exploration,
  reusing experience data, scheduling skills, and learning from observations,
  which traditionally require separate, vertically designed algorithms. We test
  our method on a sparse-reward simulated robotic manipulation environment, where
  a robot needs to stack a set of objects. We demonstrate substantial performance
  improvements over baselines in exploration efficiency and ability to reuse data
  from offline datasets, and illustrate how to reuse learned skills to solve
  novel tasks or imitate videos of human experts.
paper_ai_summary: A framework using language as a reasoning tool in RL agents improves exploration efficiency and data reuse, and enhances the acquisition of skills for novel tasks.
paper_ai_keywords: language models vision language models reinforcement learning (RL) efficient exploration experience data reuse scheduling skills learning from observations sparse-reward environment robotic manipulation offline datasets skill reuse imitating human experts