RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
Recent advances in robotic foundation models have enabled the development of
generalist policies that can adapt to diverse tasks. While these models show
impressive flexibility, their performance heavily depends on the quality of
their training data. In this work, we propose Reinforcement Learning Distilled
Generalists (RLDG), a method that leverages reinforcement learning to generate
high-quality training data for finetuning generalist policies. Through
extensive real-world experiments on precise manipulation tasks like connector
insertion and assembly, we demonstrate that generalist policies trained with
RL-generated data consistently outperform those trained with human
demonstrations, achieving up to 40% higher success rates while generalizing
better to new tasks. We also provide a detailed analysis that reveals this
performance gain stems from both optimized action distributions and improved
state coverage. Our results suggest that combining task-specific RL with
generalist policy distillation offers a promising approach for developing more
capable and efficient robotic manipulation systems that maintain the
flexibility of foundation models while achieving the performance of specialized
controllers. Videos and code can be found on our project website
https://generalist-distillation.github.io
6762120bc29af18b3fed8846
Charles Xu
extracted_pending
2024-12-18T00:06:36.669Z
658fdaacac02633c0d78e3e8
/avatars/c63481494f86368185a9e33587de77d3.svg
Charles Xu
user
charlesxu0124
6762120bc29af18b3fed8847
Qiyang Li
6762120bc29af18b3fed8848
Jianlan Luo
extracted_confirmed
2024-12-18T00:13:02.332Z
64f8cb8ed04a890f5380d9a4
/avatars/d6fdfdbb0c10141aa3b4c832d928121b.svg
Jianlan Luo
user
jianlanluo
6762120bc29af18b3fed8849
Sergey Levine
RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
Recent advances in robotic foundation models have enabled the development of
generalist policies that can adapt to diverse tasks. While these models show
impressive flexibility, their performance heavily depends on the quality of
their training data. In this work, we propose Reinforcement Learning Distilled
Generalists (RLDG), a method that leverages reinforcement learning to generate
high-quality training data for finetuning generalist policies. Through
extensive real-world experiments on precise manipulation tasks like connector
insertion and assembly, we demonstrate that generalist policies trained with
RL-generated data consistently outperform those trained with human
demonstrations, achieving up to 40% higher success rates while generalizing
better to new tasks. We also provide a detailed analysis that reveals this
performance gain stems from both optimized action distributions and improved
state coverage. Our results suggest that combining task-specific RL with
generalist policy distillation offers a promising approach for developing more
capable and efficient robotic manipulation systems that maintain the
flexibility of foundation models while achieving the performance of specialized
controllers. Videos and code can be found on our project website
https://generalist-distillation.github.io
Reinforcement Learning Distilled Generalists (RLDG) enhances robotic generalist policies by generating high-quality training data, resulting in superior performance and better task generalization compared to policies trained with human demonstrations.
Reinforcement Learning Distilled Generalists RLDG reinforcement learning generalist policies training data human demonstrations connector insertion assembly action distributions state coverage robotic manipulation