title: RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
summary: Recent advances in robotic foundation models have enabled the development of
  generalist policies that can adapt to diverse tasks. While these models show
  impressive flexibility, their performance heavily depends on the quality of
  their training data. In this work, we propose Reinforcement Learning Distilled
  Generalists (RLDG), a method that leverages reinforcement learning to generate
  high-quality training data for finetuning generalist policies. Through
  extensive real-world experiments on precise manipulation tasks like connector
  insertion and assembly, we demonstrate that generalist policies trained with
  RL-generated data consistently outperform those trained with human
  demonstrations, achieving up to 40% higher success rates while generalizing
  better to new tasks. We also provide a detailed analysis that reveals this
  performance gain stems from both optimized action distributions and improved
  state coverage. Our results suggest that combining task-specific RL with
  generalist policy distillation offers a promising approach for developing more
  capable and efficient robotic manipulation systems that maintain the
  flexibility of foundation models while achieving the performance of specialized
  controllers. Videos and code can be found on our project website
  https://generalist-distillation.github.io
paper_authors._id: 6762120bc29af18b3fed8846
paper_authors.name: Charles Xu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-18T00:06:36.669Z
paper_authors._id: 658fdaacac02633c0d78e3e8
paper_authors.avatarUrl: /avatars/c63481494f86368185a9e33587de77d3.svg
paper_authors.fullname: Charles Xu
paper_authors.type: user
paper_authors.user: charlesxu0124
paper_authors._id: 6762120bc29af18b3fed8847
paper_authors.name: Qiyang Li
paper_authors._id: 6762120bc29af18b3fed8848
paper_authors.name: Jianlan Luo
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-12-18T00:13:02.332Z
paper_authors._id: 64f8cb8ed04a890f5380d9a4
paper_authors.avatarUrl: /avatars/d6fdfdbb0c10141aa3b4c832d928121b.svg
paper_authors.fullname: Jianlan Luo
paper_authors.type: user
paper_authors.user: jianlanluo
paper_authors._id: 6762120bc29af18b3fed8849
paper_authors.name: Sergey Levine
paper_title: RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
paper_summary: Recent advances in robotic foundation models have enabled the development of
  generalist policies that can adapt to diverse tasks. While these models show
  impressive flexibility, their performance heavily depends on the quality of
  their training data. In this work, we propose Reinforcement Learning Distilled
  Generalists (RLDG), a method that leverages reinforcement learning to generate
  high-quality training data for finetuning generalist policies. Through
  extensive real-world experiments on precise manipulation tasks like connector
  insertion and assembly, we demonstrate that generalist policies trained with
  RL-generated data consistently outperform those trained with human
  demonstrations, achieving up to 40% higher success rates while generalizing
  better to new tasks. We also provide a detailed analysis that reveals this
  performance gain stems from both optimized action distributions and improved
  state coverage. Our results suggest that combining task-specific RL with
  generalist policy distillation offers a promising approach for developing more
  capable and efficient robotic manipulation systems that maintain the
  flexibility of foundation models while achieving the performance of specialized
  controllers. Videos and code can be found on our project website
  https://generalist-distillation.github.io
paper_ai_keywords: Reinforcement Learning Distilled Generalists (RLDG) reinforcement learning generalist policies high-quality training data finetuning real-world experiments connector insertion assembly action distributions state coverage task-specific RL generalist policy distillation robotic manipulation systems specialized controllers