Motion Mamba: Efficient and Long Sequence Motion Generation with
  Hierarchical and Bidirectional Selective SSM
Human motion generation stands as a significant pursuit in generative
computer vision, while achieving long-sequence and efficient motion generation
remains challenging. Recent advancements in state space models (SSMs), notably
Mamba, have showcased considerable promise in long sequence modeling with an
efficient hardware-aware design, which appears to be a promising direction to
build motion generation model upon it. Nevertheless, adapting SSMs to motion
generation faces hurdles since the lack of a specialized design architecture to
model motion sequence. To address these challenges, we propose Motion Mamba, a
simple and efficient approach that presents the pioneering motion generation
model utilized SSMs. Specifically, we design a Hierarchical Temporal Mamba
(HTM) block to process temporal data by ensemble varying numbers of isolated
SSM modules across a symmetric U-Net architecture aimed at preserving motion
consistency between frames. We also design a Bidirectional Spatial Mamba (BSM)
block to bidirectionally process latent poses, to enhance accurate motion
generation within a temporal frame. Our proposed method achieves up to 50% FID
improvement and up to 4 times faster on the HumanML3D and KIT-ML datasets
compared to the previous best diffusion-based method, which demonstrates strong
capabilities of high-quality long sequence motion modeling and real-time human
motion generation. See project website
https://steve-zeyu-zhang.github.io/MotionMamba/
65f11987e0aa03690e823794
Zeyu Zhang
claimed_verified
2024-07-03T07:42:15.670Z
64ec877bb93654d4ca5c92e9
https://cdn-avatars.huggingface.co/v1/production/uploads/64ec877bb93654d4ca5c92e9/GvHk_KSdE9Rhnk_o-NaZX.jpeg
Zeyu Zhang
user
SteveZeyuZhang
65f11987e0aa03690e823795
Akide Liu
admin_assigned
2024-03-13T13:51:41.537Z
63ebc290d64e6436e2311074
/avatars/13f08fbf3736e471e10bfc417377575e.svg
Akide Liu
user
Akide
65f11987e0aa03690e823796
Ian Reid
65f11987e0aa03690e823797
Richard Hartley
65f11987e0aa03690e823798
Bohan Zhuang
admin_assigned
2024-03-13T13:52:18.687Z
65a8a828ef14f9e6037dfad2
/avatars/55fdc0393b2e37ac998ddbdb1e1ff636.svg
Bohan Zhuang
user
BohanZ
65f11987e0aa03690e823799
Hao Tang
Motion Mamba: Efficient and Long Sequence Motion Generation with
  Hierarchical and Bidirectional Selective SSM
Human motion generation stands as a significant pursuit in generative
computer vision, while achieving long-sequence and efficient motion generation
remains challenging. Recent advancements in state space models (SSMs), notably
Mamba, have showcased considerable promise in long sequence modeling with an
efficient hardware-aware design, which appears to be a promising direction to
build motion generation model upon it. Nevertheless, adapting SSMs to motion
generation faces hurdles since the lack of a specialized design architecture to
model motion sequence. To address these challenges, we propose Motion Mamba, a
simple and efficient approach that presents the pioneering motion generation
model utilized SSMs. Specifically, we design a Hierarchical Temporal Mamba
(HTM) block to process temporal data by ensemble varying numbers of isolated
SSM modules across a symmetric U-Net architecture aimed at preserving motion
consistency between frames. We also design a Bidirectional Spatial Mamba (BSM)
block to bidirectionally process latent poses, to enhance accurate motion
generation within a temporal frame. Our proposed method achieves up to 50% FID
improvement and up to 4 times faster on the HumanML3D and KIT-ML datasets
compared to the previous best diffusion-based method, which demonstrates strong
capabilities of high-quality long sequence motion modeling and real-time human
motion generation. See project website
https://steve-zeyu-zhang.github.io/MotionMamba/
Motion Mamba, utilizing a hierarchical and bidirectional SSM-based architecture, enhances motion generation by improving quality and speed on HumanML3D and KIT-ML datasets.
state space models Mamba Hierarchical Temporal Mamba HTM Bidirectional Spatial Mamba BSM U-Net architecture long sequence motion modeling human motion generation FID improvement