Diffusion Augmented Agents: A Framework for Efficient Exploration and
  Transfer Learning
We introduce Diffusion Augmented Agents (DAAG), a novel framework that
leverages large language models, vision language models, and diffusion models
to improve sample efficiency and transfer learning in reinforcement learning
for embodied agents. DAAG hindsight relabels the agent's past experience by
using diffusion models to transform videos in a temporally and geometrically
consistent way to align with target instructions with a technique we call
Hindsight Experience Augmentation. A large language model orchestrates this
autonomous process without requiring human supervision, making it well-suited
for lifelong learning scenarios. The framework reduces the amount of
reward-labeled data needed to 1) finetune a vision language model that acts as
a reward detector, and 2) train RL agents on new tasks. We demonstrate the
sample efficiency gains of DAAG in simulated robotics environments involving
manipulation and navigation. Our results show that DAAG improves learning of
reward detectors, transferring past experience, and acquiring new tasks - key
abilities for developing efficient lifelong learning agents. Supplementary
material and visualizations are available on our website
https://sites.google.com/view/diffusion-augmented-agents/
66a9c389f47cae66bc0e3178
Norman Di Palo
extracted_pending
2024-07-31T04:54:37.097Z
62dac377f014388f908974f4
/avatars/39bf0ca206441575a45f577060cdd8bc.svg
Norman Di Palo
user
normandipalo
66a9c389f47cae66bc0e3179
Leonard Hasenclever
66a9c389f47cae66bc0e317a
Jan Humplik
66a9c389f47cae66bc0e317b
Arunkumar Byravan
Diffusion Augmented Agents: A Framework for Efficient Exploration and
  Transfer Learning
We introduce Diffusion Augmented Agents (DAAG), a novel framework that
leverages large language models, vision language models, and diffusion models
to improve sample efficiency and transfer learning in reinforcement learning
for embodied agents. DAAG hindsight relabels the agent's past experience by
using diffusion models to transform videos in a temporally and geometrically
consistent way to align with target instructions with a technique we call
Hindsight Experience Augmentation. A large language model orchestrates this
autonomous process without requiring human supervision, making it well-suited
for lifelong learning scenarios. The framework reduces the amount of
reward-labeled data needed to 1) finetune a vision language model that acts as
a reward detector, and 2) train RL agents on new tasks. We demonstrate the
sample efficiency gains of DAAG in simulated robotics environments involving
manipulation and navigation. Our results show that DAAG improves learning of
reward detectors, transferring past experience, and acquiring new tasks - key
abilities for developing efficient lifelong learning agents. Supplementary
material and visualizations are available on our website
https://sites.google.com/view/diffusion-augmented-agents/
Diffusion Augmented Agents (DAAG) enhances reinforcement learning for embodied agents using large and vision language models, diffusion models, and hindsight experience augmentation to improve sample efficiency and transfer learning.
diffusion models reinforcement learning embodied agents large language models vision language models hindsight relabeling hindsight experience augmentation sample efficiency transfer learning lifelong learning reward detector simulated robotics environments manipulation navigation