title: From Text to Motion: Grounding GPT-4 in a Humanoid Robot "Alter3"
summary: We report the development of Alter3, a humanoid robot capable of generating
  spontaneous motion using a Large Language Model (LLM), specifically GPT-4. This
  achievement was realized by integrating GPT-4 into our proprietary android,
  Alter3, thereby effectively grounding the LLM with Alter's bodily movement.
  Typically, low-level robot control is hardware-dependent and falls outside the
  scope of LLM corpora, presenting challenges for direct LLM-based robot control.
  However, in the case of humanoid robots like Alter3, direct control is feasible
  by mapping the linguistic expressions of human actions onto the robot's body
  through program code. Remarkably, this approach enables Alter3 to adopt various
  poses, such as a 'selfie' stance or 'pretending to be a ghost,' and generate
  sequences of actions over time without explicit programming for each body part.
  This demonstrates the robot's zero-shot learning capabilities. Additionally,
  verbal feedback can adjust poses, obviating the need for fine-tuning. A video
  of Alter3's generated motions is available at
  https://tnoinkwms.github.io/ALTER-LLM/
paper_authors._id: 6577ce7f9c5c26f96ce634ee
paper_authors.name: Takahide Yoshida
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-12T16:23:29.583Z
paper_authors._id: 6578307176b6de7978ffeee7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wRf2mV0sIGIDzdrwkH6nC.png
paper_authors.fullname: Takahide Yoshida
paper_authors.type: user
paper_authors.user: tnoinkwms
paper_authors._id: 6577ce7f9c5c26f96ce634ef
paper_authors.name: Atsushi Masumori
paper_authors._id: 6577ce7f9c5c26f96ce634f0
paper_authors.name: Takashi Ikegami
paper_title: From Text to Motion: Grounding GPT-4 in a Humanoid Robot "Alter3"
paper_summary: We report the development of Alter3, a humanoid robot capable of generating
  spontaneous motion using a Large Language Model (LLM), specifically GPT-4. This
  achievement was realized by integrating GPT-4 into our proprietary android,
  Alter3, thereby effectively grounding the LLM with Alter's bodily movement.
  Typically, low-level robot control is hardware-dependent and falls outside the
  scope of LLM corpora, presenting challenges for direct LLM-based robot control.
  However, in the case of humanoid robots like Alter3, direct control is feasible
  by mapping the linguistic expressions of human actions onto the robot's body
  through program code. Remarkably, this approach enables Alter3 to adopt various
  poses, such as a 'selfie' stance or 'pretending to be a ghost,' and generate
  sequences of actions over time without explicit programming for each body part.
  This demonstrates the robot's zero-shot learning capabilities. Additionally,
  verbal feedback can adjust poses, obviating the need for fine-tuning. A video
  of Alter3's generated motions is available at
  https://tnoinkwms.github.io/ALTER-LLM/
paper_ai_keywords: Large Language Model (LLM) GPT-4 humanoid robot low-level robot control LLM corpora program code zero-shot learning capabilities