title: Motion Anything: Any to Motion Generation
summary: Conditional motion generation has been extensively studied in computer
  vision, yet two critical challenges remain. First, while masked autoregressive
  methods have recently outperformed diffusion-based approaches, existing masking
  models lack a mechanism to prioritize dynamic frames and body parts based on
  given conditions. Second, existing methods for different conditioning
  modalities often fail to integrate multiple modalities effectively, limiting
  control and coherence in generated motion. To address these challenges, we
  propose Motion Anything, a multimodal motion generation framework that
  introduces an Attention-based Mask Modeling approach, enabling fine-grained
  spatial and temporal control over key frames and actions. Our model adaptively
  encodes multimodal conditions, including text and music, improving
  controllability. Additionally, we introduce Text-Music-Dance (TMD), a new
  motion dataset consisting of 2,153 pairs of text, music, and dance, making it
  twice the size of AIST++, thereby filling a critical gap in the community.
  Extensive experiments demonstrate that Motion Anything surpasses
  state-of-the-art methods across multiple benchmarks, achieving a 15%
  improvement in FID on HumanML3D and showing consistent performance gains on
  AIST++ and TMD. See our project website
  https://steve-zeyu-zhang.github.io/MotionAnything
paper_authors._id: 67d2748117d5cbff7c23621e
paper_authors.name: Zeyu Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-13T08:23:56.553Z
paper_authors._id: 64ec877bb93654d4ca5c92e9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ec877bb93654d4ca5c92e9/GvHk_KSdE9Rhnk_o-NaZX.jpeg
paper_authors.fullname: Zeyu Zhang
paper_authors.type: user
paper_authors.user: SteveZeyuZhang
paper_authors._id: 67d2748117d5cbff7c23621f
paper_authors.name: Yiran Wang
paper_authors._id: 67d2748117d5cbff7c236220
paper_authors.name: Wei Mao
paper_authors._id: 67d2748117d5cbff7c236221
paper_authors.name: Danning Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:20:46.406Z
paper_authors._id: 676ee04c0892540c7ad1ec42
paper_authors.avatarUrl: /avatars/6fc31c22d68e3e55c6c572c241dadfb0.svg
paper_authors.fullname: Danning Li
paper_authors.type: user
paper_authors.user: CAPYLEE
paper_authors._id: 67d2748117d5cbff7c236222
paper_authors.name: Rui Zhao
paper_authors._id: 67d2748117d5cbff7c236223
paper_authors.name: Biao Wu
paper_authors._id: 67d2748117d5cbff7c236224
paper_authors.name: Zirui Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:58:50.248Z
paper_authors._id: 65407ba7a38390065750233f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65407ba7a38390065750233f/1_IPMZbk-S9u2t18PQgMp.jpeg
paper_authors.fullname: Zirui Song
paper_authors.type: user
paper_authors.user: Ziruibest
paper_authors._id: 67d2748117d5cbff7c236225
paper_authors.name: Bohan Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:20:30.017Z
paper_authors._id: 65a8a828ef14f9e6037dfad2
paper_authors.avatarUrl: /avatars/55fdc0393b2e37ac998ddbdb1e1ff636.svg
paper_authors.fullname: Bohan Zhuang
paper_authors.type: user
paper_authors.user: BohanZ
paper_authors._id: 67d2748117d5cbff7c236226
paper_authors.name: Ian Reid
paper_authors._id: 67d2748117d5cbff7c236227
paper_authors.name: Richard Hartley
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:20:36.511Z
paper_authors._id: 67a4b34096628f7a5a39f333
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OI9pwYpPUO6c087WZR2oS.png
paper_authors.fullname: Richard Hartley
paper_authors.type: user
paper_authors.user: khobalt61
paper_title: Motion Anything: Any to Motion Generation
paper_summary: Conditional motion generation has been extensively studied in computer
  vision, yet two critical challenges remain. First, while masked autoregressive
  methods have recently outperformed diffusion-based approaches, existing masking
  models lack a mechanism to prioritize dynamic frames and body parts based on
  given conditions. Second, existing methods for different conditioning
  modalities often fail to integrate multiple modalities effectively, limiting
  control and coherence in generated motion. To address these challenges, we
  propose Motion Anything, a multimodal motion generation framework that
  introduces an Attention-based Mask Modeling approach, enabling fine-grained
  spatial and temporal control over key frames and actions. Our model adaptively
  encodes multimodal conditions, including text and music, improving
  controllability. Additionally, we introduce Text-Music-Dance (TMD), a new
  motion dataset consisting of 2,153 pairs of text, music, and dance, making it
  twice the size of AIST++, thereby filling a critical gap in the community.
  Extensive experiments demonstrate that Motion Anything surpasses
  state-of-the-art methods across multiple benchmarks, achieving a 15%
  improvement in FID on HumanML3D and showing consistent performance gains on
  AIST++ and TMD. See our project website
  https://steve-zeyu-zhang.github.io/MotionAnything
paper_ai_keywords: masked autoregressive methods diffusion-based approaches masking models dynamic frames body parts conditional motion generation multimodal motion generation framework Attention-based Mask Modeling key frames actions multimodal conditions Text-Music-Dance (TMD) FID HumanML3D AIST++