title: Scaling Cross-Embodied Learning: One Policy for Manipulation,
    Navigation, Locomotion and Aviation
summary: Modern machine learning systems rely on large datasets to attain broad
  generalization, and this often poses a challenge in robot learning, where each
  robotic platform and task might have only a small dataset. By training a single
  policy across many different kinds of robots, a robot learning method can
  leverage much broader and more diverse datasets, which in turn can lead to
  better generalization and robustness. However, training a single policy on
  multi-robot data is challenging because robots can have widely varying sensors,
  actuators, and control frequencies. We propose CrossFormer, a scalable and
  flexible transformer-based policy that can consume data from any embodiment. We
  train CrossFormer on the largest and most diverse dataset to date, 900K
  trajectories across 20 different robot embodiments. We demonstrate that the
  same network weights can control vastly different robots, including single and
  dual arm manipulation systems, wheeled robots, quadcopters, and quadrupeds.
  Unlike prior work, our model does not require manual alignment of the
  observation or action spaces. Extensive experiments in the real world show that
  our method matches the performance of specialist policies tailored for each
  embodiment, while also significantly outperforming the prior state of the art
  in cross-embodiment learning.
paper_authors._id: 66c766d98ad5f9c46f8c9bdb
paper_authors.name: Ria Doshi
paper_authors._id: 66c766d98ad5f9c46f8c9bdc
paper_authors.name: Homer Walke
paper_authors._id: 66c766d98ad5f9c46f8c9bdd
paper_authors.name: Oier Mees
paper_authors._id: 66c766d98ad5f9c46f8c9bde
paper_authors.name: Sudeep Dasari
paper_authors._id: 66c766d98ad5f9c46f8c9bdf
paper_authors.name: Sergey Levine
paper_title: Scaling Cross-Embodied Learning: One Policy for Manipulation,
    Navigation, Locomotion and Aviation
paper_summary: Modern machine learning systems rely on large datasets to attain broad
  generalization, and this often poses a challenge in robot learning, where each
  robotic platform and task might have only a small dataset. By training a single
  policy across many different kinds of robots, a robot learning method can
  leverage much broader and more diverse datasets, which in turn can lead to
  better generalization and robustness. However, training a single policy on
  multi-robot data is challenging because robots can have widely varying sensors,
  actuators, and control frequencies. We propose CrossFormer, a scalable and
  flexible transformer-based policy that can consume data from any embodiment. We
  train CrossFormer on the largest and most diverse dataset to date, 900K
  trajectories across 20 different robot embodiments. We demonstrate that the
  same network weights can control vastly different robots, including single and
  dual arm manipulation systems, wheeled robots, quadcopters, and quadrupeds.
  Unlike prior work, our model does not require manual alignment of the
  observation or action spaces. Extensive experiments in the real world show that
  our method matches the performance of specialist policies tailored for each
  embodiment, while also significantly outperforming the prior state of the art
  in cross-embodiment learning.
paper_ai_summary: CrossFormer, a transformer-based policy, demonstrates scalable and flexible control across diverse robotic embodiments using a unified training approach.
paper_ai_keywords: transformer-based policy CrossFormer multi-robot data sensor actuator control frequencies observation spaces action spaces cross-embodiment learning