title: SayTap: Language to Quadrupedal Locomotion
summary: Large language models (LLMs) have demonstrated the potential to perform
  high-level planning. Yet, it remains a challenge for LLMs to comprehend
  low-level commands, such as joint angle targets or motor torques. This paper
  proposes an approach to use foot contact patterns as an interface that bridges
  human commands in natural language and a locomotion controller that outputs
  these low-level commands. This results in an interactive system for quadrupedal
  robots that allows the users to craft diverse locomotion behaviors flexibly. We
  contribute an LLM prompt design, a reward function, and a method to expose the
  controller to the feasible distribution of contact patterns. The results are a
  controller capable of achieving diverse locomotion patterns that can be
  transferred to real robot hardware. Compared with other design choices, the
  proposed approach enjoys more than 50% success rate in predicting the correct
  contact patterns and can solve 10 more tasks out of a total of 30 tasks. Our
  project site is: https://saytap.github.io.
paper_authors._id: 64891faa7987267e4fc3f608
paper_authors.name: Yujin Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-14T12:14:58.914Z
paper_authors._id: 63285beda4102056bc198395
paper_authors.avatarUrl: /avatars/6c936818a3814c1c32a8eaae8b8cb6d6.svg
paper_authors.fullname: yujin tang
paper_authors.type: user
paper_authors.user: tyj2022
paper_authors._id: 64891faa7987267e4fc3f609
paper_authors.name: Wenhao Yu
paper_authors._id: 64891faa7987267e4fc3f60a
paper_authors.name: Jie Tan
paper_authors._id: 64891faa7987267e4fc3f60b
paper_authors.name: Heiga Zen
paper_authors._id: 64891faa7987267e4fc3f60c
paper_authors.name: Aleksandra Faust
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T07:24:07.283Z
paper_authors._id: 64a3b828225b2121f6b3beb3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64a3b828225b2121f6b3beb3/44AgAQENc7cJMOWcoK4g_.jpeg
paper_authors.fullname: Aleksandra Faust
paper_authors.type: user
paper_authors.user: sandraorion
paper_authors._id: 64891faa7987267e4fc3f60d
paper_authors.name: Tatsuya Harada
paper_title: SayTap: Language to Quadrupedal Locomotion
paper_summary: Large language models (LLMs) have demonstrated the potential to perform
  high-level planning. Yet, it remains a challenge for LLMs to comprehend
  low-level commands, such as joint angle targets or motor torques. This paper
  proposes an approach to use foot contact patterns as an interface that bridges
  human commands in natural language and a locomotion controller that outputs
  these low-level commands. This results in an interactive system for quadrupedal
  robots that allows the users to craft diverse locomotion behaviors flexibly. We
  contribute an LLM prompt design, a reward function, and a method to expose the
  controller to the feasible distribution of contact patterns. The results are a
  controller capable of achieving diverse locomotion patterns that can be
  transferred to real robot hardware. Compared with other design choices, the
  proposed approach enjoys more than 50% success rate in predicting the correct
  contact patterns and can solve 10 more tasks out of a total of 30 tasks. Our
  project site is: https://saytap.github.io.
paper_ai_keywords: large language models (LLMs) high-level planning low-level commands joint angle targets motor torques foot contact patterns natural language locomotion controller quadrupedal robots prompt design reward function feasible distribution of contact patterns locomotion patterns real robot hardware