MotionGPT: Finetuned LLMs are General-Purpose Motion Generators
Generating realistic human motion from given action descriptions has
experienced significant advancements because of the emerging requirement of
digital humans. While recent works have achieved impressive results in
generating motion directly from textual action descriptions, they often support
only a single modality of the control signal, which limits their application in
the real digital human industry. This paper presents a Motion General-Purpose
generaTor (MotionGPT) that can use multimodal control signals, e.g., text and
single-frame poses, for generating consecutive human motions by treating
multimodal signals as special input tokens in large language models (LLMs).
Specifically, we first quantize multimodal control signals into discrete codes
and then formulate them in a unified prompt instruction to ask the LLMs to
generate the motion answer. Our MotionGPT demonstrates a unified human motion
generation model with multimodal control signals by tuning a mere 0.4% of LLM
parameters. To the best of our knowledge, MotionGPT is the first method to
generate human motion by multimodal control signals, which we hope can shed
light on this new direction. Codes shall be released upon acceptance.
64928774d03573a0d593e38d
Yaqi Zhang
64928774d03573a0d593e38e
Di Huang
64928774d03573a0d593e38f
Bin Liu
64928774d03573a0d593e390
Shixiang Tang
64928774d03573a0d593e391
Yan Lu
64928774d03573a0d593e392
Lu Chen
64928774d03573a0d593e393
Lei Bai
64928774d03573a0d593e394
Qi Chu
64928774d03573a0d593e395
Nenghai Yu
64928774d03573a0d593e396
Wanli Ouyang
MotionGPT: Finetuned LLMs are General-Purpose Motion Generators
Generating realistic human motion from given action descriptions has
experienced significant advancements because of the emerging requirement of
digital humans. While recent works have achieved impressive results in
generating motion directly from textual action descriptions, they often support
only a single modality of the control signal, which limits their application in
the real digital human industry. This paper presents a Motion General-Purpose
generaTor (MotionGPT) that can use multimodal control signals, e.g., text and
single-frame poses, for generating consecutive human motions by treating
multimodal signals as special input tokens in large language models (LLMs).
Specifically, we first quantize multimodal control signals into discrete codes
and then formulate them in a unified prompt instruction to ask the LLMs to
generate the motion answer. Our MotionGPT demonstrates a unified human motion
generation model with multimodal control signals by tuning a mere 0.4% of LLM
parameters. To the best of our knowledge, MotionGPT is the first method to
generate human motion by multimodal control signals, which we hope can shed
light on this new direction. Codes shall be released upon acceptance.
MotionGPT generates realistic human motion using multimodal control signals by adapting a small portion of LLM parameters to handle unified input instructions.
Motion General-Purpose generaTor MotionGPT multimodal control signals discrete codes unified prompt instruction large language models LLM parameter-efficient fine-tuning