Motion Anything: Any to Motion Generation
Conditional motion generation has been extensively studied in computer
vision, yet two critical challenges remain. First, while masked autoregressive
methods have recently outperformed diffusion-based approaches, existing masking
models lack a mechanism to prioritize dynamic frames and body parts based on
given conditions. Second, existing methods for different conditioning
modalities often fail to integrate multiple modalities effectively, limiting
control and coherence in generated motion. To address these challenges, we
propose Motion Anything, a multimodal motion generation framework that
introduces an Attention-based Mask Modeling approach, enabling fine-grained
spatial and temporal control over key frames and actions. Our model adaptively
encodes multimodal conditions, including text and music, improving
controllability. Additionally, we introduce Text-Music-Dance (TMD), a new
motion dataset consisting of 2,153 pairs of text, music, and dance, making it
twice the size of AIST++, thereby filling a critical gap in the community.
Extensive experiments demonstrate that Motion Anything surpasses
state-of-the-art methods across multiple benchmarks, achieving a 15%
improvement in FID on HumanML3D and showing consistent performance gains on
AIST++ and TMD. See our project website
https://steve-zeyu-zhang.github.io/MotionAnything
67d2748117d5cbff7c23621e
Zeyu Zhang
claimed_verified
2025-03-13T08:23:56.553Z
64ec877bb93654d4ca5c92e9
https://cdn-avatars.huggingface.co/v1/production/uploads/64ec877bb93654d4ca5c92e9/GvHk_KSdE9Rhnk_o-NaZX.jpeg
Zeyu Zhang
user
SteveZeyuZhang
67d2748117d5cbff7c23621f
Yiran Wang
67d2748117d5cbff7c236220
Wei Mao
67d2748117d5cbff7c236221
Danning Li
admin_assigned
2025-03-14T11:20:46.406Z
676ee04c0892540c7ad1ec42
/avatars/6fc31c22d68e3e55c6c572c241dadfb0.svg
Danning Li
user
CAPYLEE
67d2748117d5cbff7c236222
Rui Zhao
67d2748117d5cbff7c236223
Biao Wu
67d2748117d5cbff7c236224
Zirui Song
claimed_verified
2025-03-14T08:58:50.248Z
65407ba7a38390065750233f
https://cdn-avatars.huggingface.co/v1/production/uploads/65407ba7a38390065750233f/1_IPMZbk-S9u2t18PQgMp.jpeg
Zirui Song
user
Ziruibest
67d2748117d5cbff7c236225
Bohan Zhuang
admin_assigned
2025-03-14T11:20:30.017Z
65a8a828ef14f9e6037dfad2
/avatars/55fdc0393b2e37ac998ddbdb1e1ff636.svg
Bohan Zhuang
user
BohanZ
67d2748117d5cbff7c236226
Ian Reid
67d2748117d5cbff7c236227
Richard Hartley
admin_assigned
2025-03-14T11:20:36.511Z
67a4b34096628f7a5a39f333
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OI9pwYpPUO6c087WZR2oS.png
Richard Hartley
user
khobalt61
Motion Anything: Any to Motion Generation
Conditional motion generation has been extensively studied in computer
vision, yet two critical challenges remain. First, while masked autoregressive
methods have recently outperformed diffusion-based approaches, existing masking
models lack a mechanism to prioritize dynamic frames and body parts based on
given conditions. Second, existing methods for different conditioning
modalities often fail to integrate multiple modalities effectively, limiting
control and coherence in generated motion. To address these challenges, we
propose Motion Anything, a multimodal motion generation framework that
introduces an Attention-based Mask Modeling approach, enabling fine-grained
spatial and temporal control over key frames and actions. Our model adaptively
encodes multimodal conditions, including text and music, improving
controllability. Additionally, we introduce Text-Music-Dance (TMD), a new
motion dataset consisting of 2,153 pairs of text, music, and dance, making it
twice the size of AIST++, thereby filling a critical gap in the community.
Extensive experiments demonstrate that Motion Anything surpasses
state-of-the-art methods across multiple benchmarks, achieving a 15%
improvement in FID on HumanML3D and showing consistent performance gains on
AIST++ and TMD. See our project website
https://steve-zeyu-zhang.github.io/MotionAnything
Motion Anything improves multimodal motion generation through attention-based masking and adaptive condition encoding, achieving state-of-the-art results across benchmarks.
masked autoregressive methods diffusion-based approaches multimodal motion generation Attention-based Mask Modeling HumanML3D AIST++ Text-Music-Dance (TMD) FID