title: MotionGPT: Finetuned LLMs are General-Purpose Motion Generators
summary: Generating realistic human motion from given action descriptions has
  experienced significant advancements because of the emerging requirement of
  digital humans. While recent works have achieved impressive results in
  generating motion directly from textual action descriptions, they often support
  only a single modality of the control signal, which limits their application in
  the real digital human industry. This paper presents a Motion General-Purpose
  generaTor (MotionGPT) that can use multimodal control signals, e.g., text and
  single-frame poses, for generating consecutive human motions by treating
  multimodal signals as special input tokens in large language models (LLMs).
  Specifically, we first quantize multimodal control signals into discrete codes
  and then formulate them in a unified prompt instruction to ask the LLMs to
  generate the motion answer. Our MotionGPT demonstrates a unified human motion
  generation model with multimodal control signals by tuning a mere 0.4% of LLM
  parameters. To the best of our knowledge, MotionGPT is the first method to
  generate human motion by multimodal control signals, which we hope can shed
  light on this new direction. Codes shall be released upon acceptance.
paper_authors._id: 64928774d03573a0d593e38d
paper_authors.name: Yaqi Zhang
paper_authors._id: 64928774d03573a0d593e38e
paper_authors.name: Di Huang
paper_authors._id: 64928774d03573a0d593e38f
paper_authors.name: Bin Liu
paper_authors._id: 64928774d03573a0d593e390
paper_authors.name: Shixiang Tang
paper_authors._id: 64928774d03573a0d593e391
paper_authors.name: Yan Lu
paper_authors._id: 64928774d03573a0d593e392
paper_authors.name: Lu Chen
paper_authors._id: 64928774d03573a0d593e393
paper_authors.name: Lei Bai
paper_authors._id: 64928774d03573a0d593e394
paper_authors.name: Qi Chu
paper_authors._id: 64928774d03573a0d593e395
paper_authors.name: Nenghai Yu
paper_authors._id: 64928774d03573a0d593e396
paper_authors.name: Wanli Ouyang
paper_title: MotionGPT: Finetuned LLMs are General-Purpose Motion Generators
paper_summary: Generating realistic human motion from given action descriptions has
  experienced significant advancements because of the emerging requirement of
  digital humans. While recent works have achieved impressive results in
  generating motion directly from textual action descriptions, they often support
  only a single modality of the control signal, which limits their application in
  the real digital human industry. This paper presents a Motion General-Purpose
  generaTor (MotionGPT) that can use multimodal control signals, e.g., text and
  single-frame poses, for generating consecutive human motions by treating
  multimodal signals as special input tokens in large language models (LLMs).
  Specifically, we first quantize multimodal control signals into discrete codes
  and then formulate them in a unified prompt instruction to ask the LLMs to
  generate the motion answer. Our MotionGPT demonstrates a unified human motion
  generation model with multimodal control signals by tuning a mere 0.4% of LLM
  parameters. To the best of our knowledge, MotionGPT is the first method to
  generate human motion by multimodal control signals, which we hope can shed
  light on this new direction. Codes shall be released upon acceptance.
paper_ai_summary: MotionGPT generates realistic human motion using multimodal control signals by adapting a small portion of LLM parameters to handle unified input instructions.
paper_ai_keywords: Motion General-Purpose generaTor MotionGPT multimodal control signals discrete codes unified prompt instruction large language models LLM parameter-efficient fine-tuning