title: Story-to-Motion: Synthesizing Infinite and Controllable Character
    Animation from Long Text
summary: Generating natural human motion from a story has the potential to transform
  the landscape of animation, gaming, and film industries. A new and challenging
  task, Story-to-Motion, arises when characters are required to move to various
  locations and perform specific motions based on a long text description. This
  task demands a fusion of low-level control (trajectories) and high-level
  control (motion semantics). Previous works in character control and
  text-to-motion have addressed related aspects, yet a comprehensive solution
  remains elusive: character control methods do not handle text description,
  whereas text-to-motion methods lack position constraints and often produce
  unstable motions. In light of these limitations, we propose a novel system that
  generates controllable, infinitely long motions and trajectories aligned with
  the input text. (1) We leverage contemporary Large Language Models to act as a
  text-driven motion scheduler to extract a series of (text, position, duration)
  pairs from long text. (2) We develop a text-driven motion retrieval scheme that
  incorporates motion matching with motion semantic and trajectory constraints.
  (3) We design a progressive mask transformer that addresses common artifacts in
  the transition motion such as unnatural pose and foot sliding. Beyond its
  pioneering role as the first comprehensive solution for Story-to-Motion, our
  system undergoes evaluation across three distinct sub-tasks: trajectory
  following, temporal action composition, and motion blending, where it
  outperforms previous state-of-the-art motion synthesis methods across the
  board. Homepage: https://story2motion.github.io/.
paper_authors._id: 6552ef3b73a5a6f9385f8fae
paper_authors.name: Zhongfei Qing
paper_authors._id: 6552ef3b73a5a6f9385f8faf
paper_authors.name: Zhongang Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-14T10:36:25.229Z
paper_authors._id: 652d06833b5997ed71ce5c46
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/xZTXEcnEogEmBm_ledJQr.jpeg
paper_authors.fullname: Zhongang Cai
paper_authors.type: user
paper_authors.user: caizhongang
paper_authors._id: 6552ef3b73a5a6f9385f8fb0
paper_authors.name: Zhitao Yang
paper_authors._id: 6552ef3b73a5a6f9385f8fb1
paper_authors.name: Lei Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T08:06:57.569Z
paper_authors._id: 6626a471430a124253f197c8
paper_authors.avatarUrl: /avatars/f5747fdbe495d1296fed9d16d8c95857.svg
paper_authors.fullname: yl-1993
paper_authors.type: user
paper_authors.user: yl-1993
paper_title: Story-to-Motion: Synthesizing Infinite and Controllable Character
    Animation from Long Text
paper_summary: Generating natural human motion from a story has the potential to transform
  the landscape of animation, gaming, and film industries. A new and challenging
  task, Story-to-Motion, arises when characters are required to move to various
  locations and perform specific motions based on a long text description. This
  task demands a fusion of low-level control (trajectories) and high-level
  control (motion semantics). Previous works in character control and
  text-to-motion have addressed related aspects, yet a comprehensive solution
  remains elusive: character control methods do not handle text description,
  whereas text-to-motion methods lack position constraints and often produce
  unstable motions. In light of these limitations, we propose a novel system that
  generates controllable, infinitely long motions and trajectories aligned with
  the input text. (1) We leverage contemporary Large Language Models to act as a
  text-driven motion scheduler to extract a series of (text, position, duration)
  pairs from long text. (2) We develop a text-driven motion retrieval scheme that
  incorporates motion matching with motion semantic and trajectory constraints.
  (3) We design a progressive mask transformer that addresses common artifacts in
  the transition motion such as unnatural pose and foot sliding. Beyond its
  pioneering role as the first comprehensive solution for Story-to-Motion, our
  system undergoes evaluation across three distinct sub-tasks: trajectory
  following, temporal action composition, and motion blending, where it
  outperforms previous state-of-the-art motion synthesis methods across the
  board. Homepage: https://story2motion.github.io/.
paper_ai_summary: A new system combines language models and motion retrieval to generate long, stable, and semantically aligned human motions from text descriptions, outperforming previous methods in trajectory, action, and blending tasks.
paper_ai_keywords: Large Language Models motion scheduler text-driven motion retrieval motion matching motion semantic trajectory constraints progressive mask transformer unnatural pose foot sliding trajectory following temporal action composition motion blending