title: RT-H: Action Hierarchies Using Language
summary: Language provides a way to break down complex concepts into digestible
  pieces. Recent works in robot imitation learning use language-conditioned
  policies that predict actions given visual observations and the high-level task
  specified in language. These methods leverage the structure of natural language
  to share data between semantically similar tasks (e.g., "pick coke can" and
  "pick an apple") in multi-task datasets. However, as tasks become more
  semantically diverse (e.g., "pick coke can" and "pour cup"), sharing data
  between tasks becomes harder, so learning to map high-level tasks to actions
  requires much more demonstration data. To bridge tasks and actions, our insight
  is to teach the robot the language of actions, describing low-level motions
  with more fine-grained phrases like "move arm forward". Predicting these
  language motions as an intermediate step between tasks and actions forces the
  policy to learn the shared structure of low-level motions across seemingly
  disparate tasks. Furthermore, a policy that is conditioned on language motions
  can easily be corrected during execution through human-specified language
  motions. This enables a new paradigm for flexible policies that can learn from
  human intervention in language. Our method RT-H builds an action hierarchy
  using language motions: it first learns to predict language motions, and
  conditioned on this and the high-level task, it predicts actions, using visual
  context at all stages. We show that RT-H leverages this language-action
  hierarchy to learn policies that are more robust and flexible by effectively
  tapping into multi-task datasets. We show that these policies not only allow
  for responding to language interventions, but can also learn from such
  interventions and outperform methods that learn from teleoperated
  interventions. Our website and videos are found at
  https://rt-hierarchy.github.io.
paper_authors._id: 65e7561fe35dc17c217bd940
paper_authors.name: Suneel Belkhale
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-31T07:45:28.008Z
paper_authors._id: 66a922488f5cada4a6cb7567
paper_authors.avatarUrl: /avatars/eab42b6ed39a0211c7ac13c88e39d7b2.svg
paper_authors.fullname: Suneel Belkhale
paper_authors.type: user
paper_authors.user: belkhale
paper_authors._id: 65e7561fe35dc17c217bd941
paper_authors.name: Tianli Ding
paper_authors._id: 65e7561fe35dc17c217bd942
paper_authors.name: Ted Xiao
paper_authors._id: 65e7561fe35dc17c217bd943
paper_authors.name: Pierre Sermanet
paper_authors._id: 65e7561fe35dc17c217bd944
paper_authors.name: Quon Vuong
paper_authors._id: 65e7561fe35dc17c217bd945
paper_authors.name: Jonathan Tompson
paper_authors._id: 65e7561fe35dc17c217bd946
paper_authors.name: Yevgen Chebotar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T21:03:13.353Z
paper_authors._id: 64c93f9c9471c54a9438162c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c93f9c9471c54a9438162c/bYGUqFsF67VuAuXecjxjX.jpeg
paper_authors.fullname: Yevgen Chebotar
paper_authors.type: user
paper_authors.user: ychebotar
paper_authors._id: 65e7561fe35dc17c217bd947
paper_authors.name: Debidatta Dwibedi
paper_authors._id: 65e7561fe35dc17c217bd948
paper_authors.name: Dorsa Sadigh
paper_title: RT-H: Action Hierarchies Using Language
paper_summary: Language provides a way to break down complex concepts into digestible
  pieces. Recent works in robot imitation learning use language-conditioned
  policies that predict actions given visual observations and the high-level task
  specified in language. These methods leverage the structure of natural language
  to share data between semantically similar tasks (e.g., "pick coke can" and
  "pick an apple") in multi-task datasets. However, as tasks become more
  semantically diverse (e.g., "pick coke can" and "pour cup"), sharing data
  between tasks becomes harder, so learning to map high-level tasks to actions
  requires much more demonstration data. To bridge tasks and actions, our insight
  is to teach the robot the language of actions, describing low-level motions
  with more fine-grained phrases like "move arm forward". Predicting these
  language motions as an intermediate step between tasks and actions forces the
  policy to learn the shared structure of low-level motions across seemingly
  disparate tasks. Furthermore, a policy that is conditioned on language motions
  can easily be corrected during execution through human-specified language
  motions. This enables a new paradigm for flexible policies that can learn from
  human intervention in language. Our method RT-H builds an action hierarchy
  using language motions: it first learns to predict language motions, and
  conditioned on this and the high-level task, it predicts actions, using visual
  context at all stages. We show that RT-H leverages this language-action
  hierarchy to learn policies that are more robust and flexible by effectively
  tapping into multi-task datasets. We show that these policies not only allow
  for responding to language interventions, but can also learn from such
  interventions and outperform methods that learn from teleoperated
  interventions. Our website and videos are found at
  https://rt-hierarchy.github.io.
paper_ai_summary: A method RT-H uses language-motion hierarchies to improve robot imitation learning by predicting low-level actions from high-level language tasks, enhancing flexibility and robustness.
paper_ai_keywords: language-conditioned policies action prediction visual observations multi-task datasets semantic diversity language of actions low-level motions language motions action hierarchy human-specified language motions language interventions teleoperated interventions