Language to Rewards for Robotic Skill Synthesis
Large language models (LLMs) have demonstrated exciting progress in acquiring
diverse new capabilities through in-context learning, ranging from logical
reasoning to code-writing. Robotics researchers have also explored using LLMs
to advance the capabilities of robotic control. However, since low-level robot
actions are hardware-dependent and underrepresented in LLM training corpora,
existing efforts in applying LLMs to robotics have largely treated LLMs as
semantic planners or relied on human-engineered control primitives to interface
with the robot. On the other hand, reward functions are shown to be flexible
representations that can be optimized for control policies to achieve diverse
tasks, while their semantic richness makes them suitable to be specified by
LLMs. In this work, we introduce a new paradigm that harnesses this realization
by utilizing LLMs to define reward parameters that can be optimized and
accomplish variety of robotic tasks. Using reward as the intermediate interface
generated by LLMs, we can effectively bridge the gap between high-level
language instructions or corrections to low-level robot actions. Meanwhile,
combining this with a real-time optimizer, MuJoCo MPC, empowers an interactive
behavior creation experience where users can immediately observe the results
and provide feedback to the system. To systematically evaluate the performance
of our proposed method, we designed a total of 17 tasks for a simulated
quadruped robot and a dexterous manipulator robot. We demonstrate that our
proposed method reliably tackles 90% of the designed tasks, while a baseline
using primitive skills as the interface with Code-as-policies achieves 50% of
the tasks. We further validated our method on a real robot arm where complex
manipulation skills such as non-prehensile pushing emerge through our
interactive system.
648bcdb4a7bfa231b27faab6
Wenhao Yu
648bcdb4a7bfa231b27faab7
Nimrod Gileadi
648bcdb4a7bfa231b27faab8
Chuyuan Fu
648bcdb4a7bfa231b27faab9
Sean Kirmani
claimed_verified
2023-06-16T17:38:34.340Z
648c85f33ffc11989bddcce1
/avatars/bdde73b95a36e32cb2975656cca46022.svg
Sean Kirmani
user
skirmani
648bcdb4a7bfa231b27faaba
Kuang-Huei Lee
648bcdb4a7bfa231b27faabb
Montse Gonzalez Arenas
648bcdb4a7bfa231b27faabc
Hao-Tien Lewis Chiang
648bcdb4a7bfa231b27faabd
Tom Erez
648bcdb4a7bfa231b27faabe
Leonard Hasenclever
648bcdb4a7bfa231b27faabf
Jan Humplik
648bcdb4a7bfa231b27faac0
Brian Ichter
648bcdb4a7bfa231b27faac1
Ted Xiao
648bcdb4a7bfa231b27faac2
Peng Xu
648bcdb4a7bfa231b27faac3
Andy Zeng
648bcdb4a7bfa231b27faac4
Tingnan Zhang
648bcdb4a7bfa231b27faac5
Nicolas Heess
648bcdb4a7bfa231b27faac6
Dorsa Sadigh
648bcdb4a7bfa231b27faac7
Jie Tan
648bcdb4a7bfa231b27faac8
Yuval Tassa
648bcdb4a7bfa231b27faac9
Fei Xia
claimed_verified
2023-06-16T13:57:17.803Z
62cfa9ee9b96f22525c36d47
/avatars/8bfe3bf63f0609b3b6b7d7c5d7f6b3e9.svg
Fei Xia
user
fxia22
Language to Rewards for Robotic Skill Synthesis
Large language models (LLMs) have demonstrated exciting progress in acquiring
diverse new capabilities through in-context learning, ranging from logical
reasoning to code-writing. Robotics researchers have also explored using LLMs
to advance the capabilities of robotic control. However, since low-level robot
actions are hardware-dependent and underrepresented in LLM training corpora,
existing efforts in applying LLMs to robotics have largely treated LLMs as
semantic planners or relied on human-engineered control primitives to interface
with the robot. On the other hand, reward functions are shown to be flexible
representations that can be optimized for control policies to achieve diverse
tasks, while their semantic richness makes them suitable to be specified by
LLMs. In this work, we introduce a new paradigm that harnesses this realization
by utilizing LLMs to define reward parameters that can be optimized and
accomplish variety of robotic tasks. Using reward as the intermediate interface
generated by LLMs, we can effectively bridge the gap between high-level
language instructions or corrections to low-level robot actions. Meanwhile,
combining this with a real-time optimizer, MuJoCo MPC, empowers an interactive
behavior creation experience where users can immediately observe the results
and provide feedback to the system. To systematically evaluate the performance
of our proposed method, we designed a total of 17 tasks for a simulated
quadruped robot and a dexterous manipulator robot. We demonstrate that our
proposed method reliably tackles 90% of the designed tasks, while a baseline
using primitive skills as the interface with Code-as-policies achieves 50% of
the tasks. We further validated our method on a real robot arm where complex
manipulation skills such as non-prehensile pushing emerge through our
interactive system.
A new method uses large language models to define reward parameters for control policies, bridging high-level language instructions to low-level robotic actions through an interactive system.
large language models in-context learning logical reasoning code-writing robotic control low-level robot actions LLM training corpora reward functions control policies semantic planners human-engineered control primitives MuJoCo MPC simulated quadruped robot dexterous manipulator robot real robot arm non-prehensile pushing