title: Octo: An Open-Source Generalist Robot Policy
summary: Large policies pretrained on diverse robot datasets have the potential to
  transform robotic learning: instead of training new policies from scratch, such
  generalist robot policies may be finetuned with only a little in-domain data,
  yet generalize broadly. However, to be widely applicable across a range of
  robotic learning scenarios, environments, and tasks, such policies need to
  handle diverse sensors and action spaces, accommodate a variety of commonly
  used robotic platforms, and finetune readily and efficiently to new domains. In
  this work, we aim to lay the groundwork for developing open-source, widely
  applicable, generalist policies for robotic manipulation. As a first step, we
  introduce Octo, a large transformer-based policy trained on 800k trajectories
  from the Open X-Embodiment dataset, the largest robot manipulation dataset to
  date. It can be instructed via language commands or goal images and can be
  effectively finetuned to robot setups with new sensory inputs and action spaces
  within a few hours on standard consumer GPUs. In experiments across 9 robotic
  platforms, we demonstrate that Octo serves as a versatile policy initialization
  that can be effectively finetuned to new observation and action spaces. We also
  perform detailed ablations of design decisions for the Octo model, from
  architecture to training data, to guide future research on building generalist
  robot models.
paper_authors._id: 664c12dcae011af8d11e07dc
paper_authors.name: Octo Model Team
paper_authors._id: 664c12dcae011af8d11e07dd
paper_authors.name: Dibya Ghosh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T08:58:21.835Z
paper_authors._id: 651b098f2930812657b90cce
paper_authors.avatarUrl: /avatars/8b93a87924316866fffc6acf10a41505.svg
paper_authors.fullname: Dibya Ghosh
paper_authors.type: user
paper_authors.user: dibyaghosh
paper_authors._id: 664c12dcae011af8d11e07de
paper_authors.name: Homer Walke
paper_authors._id: 664c12dcae011af8d11e07df
paper_authors.name: Karl Pertsch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T08:58:37.919Z
paper_authors._id: 65d4c1ff29b4ac81c265e6e6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65d4c1ff29b4ac81c265e6e6/GXgs28okxGfpBdqhxov9-.png
paper_authors.fullname: Karl Pertsch
paper_authors.type: user
paper_authors.user: KarlP
paper_authors._id: 664c12dcae011af8d11e07e0
paper_authors.name: Kevin Black
paper_authors._id: 664c12dcae011af8d11e07e1
paper_authors.name: Oier Mees
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T08:59:25.660Z
paper_authors._id: 663a7190d9dee283e3f56150
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663a7190d9dee283e3f56150/zoYBdFIQPGWTS0R7bpg07.jpeg
paper_authors.fullname: Oier Mees
paper_authors.type: user
paper_authors.user: oier-mees
paper_authors._id: 664c12dcae011af8d11e07e2
paper_authors.name: Sudeep Dasari
paper_authors._id: 664c12dcae011af8d11e07e3
paper_authors.name: Joey Hejna
paper_authors._id: 664c12dcae011af8d11e07e4
paper_authors.name: Tobias Kreiman
paper_authors._id: 664c12dcae011af8d11e07e5
paper_authors.name: Charles Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T09:00:26.427Z
paper_authors._id: 658fdaacac02633c0d78e3e8
paper_authors.avatarUrl: /avatars/c63481494f86368185a9e33587de77d3.svg
paper_authors.fullname: Charles Xu
paper_authors.type: user
paper_authors.user: charlesxu0124
paper_authors._id: 664c12dcae011af8d11e07e6
paper_authors.name: Jianlan Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T09:01:01.937Z
paper_authors._id: 64f8cb8ed04a890f5380d9a4
paper_authors.avatarUrl: /avatars/d6fdfdbb0c10141aa3b4c832d928121b.svg
paper_authors.fullname: Jianlan Luo
paper_authors.type: user
paper_authors.user: jianlanluo
paper_authors._id: 664c12dcae011af8d11e07e7
paper_authors.name: You Liang Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T09:01:23.423Z
paper_authors._id: 635dcbee79599a4f7db46db3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/635dcbee79599a4f7db46db3/eIi5cPQelHi0TkNNe1_XT.jpeg
paper_authors.fullname: You Liang Tan
paper_authors.type: user
paper_authors.user: youliangtan
paper_authors._id: 664c12dcae011af8d11e07e8
paper_authors.name: Pannag Sanketi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-09T07:48:53.973Z
paper_authors._id: 66b56f73fc66a466e6104bfb
paper_authors.avatarUrl: /avatars/74855aaf580144fafead4b93d6a24a46.svg
paper_authors.fullname: Pannag Sanketi
paper_authors.type: user
paper_authors.user: prsanky
paper_authors._id: 664c12dcae011af8d11e07e9
paper_authors.name: Quan Vuong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T09:01:35.171Z
paper_authors._id: 63e1d6687fbb6ae4d4f4d980
paper_authors.avatarUrl: /avatars/4959319e69d69e03a31ff4be326e602a.svg
paper_authors.fullname: Quan Vuong
paper_authors.type: user
paper_authors.user: kabut1211
paper_authors._id: 664c12dcae011af8d11e07ea
paper_authors.name: Ted Xiao
paper_authors._id: 664c12dcae011af8d11e07eb
paper_authors.name: Dorsa Sadigh
paper_authors._id: 664c12dcae011af8d11e07ec
paper_authors.name: Chelsea Finn
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-21T09:02:07.627Z
paper_authors._id: 64ac22a9193f0a807deb673d
paper_authors.avatarUrl: /avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
paper_authors.fullname: Chelsea Finn
paper_authors.type: user
paper_authors.user: cbfinn
paper_authors._id: 664c12dcae011af8d11e07ed
paper_authors.name: Sergey Levine
paper_title: Octo: An Open-Source Generalist Robot Policy
paper_summary: Large policies pretrained on diverse robot datasets have the potential to
  transform robotic learning: instead of training new policies from scratch, such
  generalist robot policies may be finetuned with only a little in-domain data,
  yet generalize broadly. However, to be widely applicable across a range of
  robotic learning scenarios, environments, and tasks, such policies need to
  handle diverse sensors and action spaces, accommodate a variety of commonly
  used robotic platforms, and finetune readily and efficiently to new domains. In
  this work, we aim to lay the groundwork for developing open-source, widely
  applicable, generalist policies for robotic manipulation. As a first step, we
  introduce Octo, a large transformer-based policy trained on 800k trajectories
  from the Open X-Embodiment dataset, the largest robot manipulation dataset to
  date. It can be instructed via language commands or goal images and can be
  effectively finetuned to robot setups with new sensory inputs and action spaces
  within a few hours on standard consumer GPUs. In experiments across 9 robotic
  platforms, we demonstrate that Octo serves as a versatile policy initialization
  that can be effectively finetuned to new observation and action spaces. We also
  perform detailed ablations of design decisions for the Octo model, from
  architecture to training data, to guide future research on building generalist
  robot models.
paper_ai_keywords: transformer-based policy Open X-Embodiment dataset language commands goal images sensory inputs action spaces policy initialization architecture training data