MotionLCM: Real-time Controllable Motion Generation via Latent
  Consistency Model
This work introduces MotionLCM, extending controllable motion generation to a
real-time level. Existing methods for spatial control in text-conditioned
motion generation suffer from significant runtime inefficiency. To address this
issue, we first propose the motion latent consistency model (MotionLCM) for
motion generation, building upon the latent diffusion model (MLD). By employing
one-step (or few-step) inference, we further improve the runtime efficiency of
the motion latent diffusion model for motion generation. To ensure effective
controllability, we incorporate a motion ControlNet within the latent space of
MotionLCM and enable explicit control signals (e.g., pelvis trajectory) in the
vanilla motion space to control the generation process directly, similar to
controlling other latent-free diffusion models for motion generation. By
employing these techniques, our approach can generate human motions with text
and control signals in real-time. Experimental results demonstrate the
remarkable generation and controlling capabilities of MotionLCM while
maintaining real-time runtime efficiency.
6631c8aa86db0efb4076f139
Wenxun Dai
claimed_verified
2024-05-02T09:04:04.779Z
6165514db5ec555e8e9c203b
https://cdn-avatars.huggingface.co/v1/production/uploads/6165514db5ec555e8e9c203b/mBdEDqRcuzIOEf3Okq6y7.png
Dai-Wenxun
user
wxDai
6631c8aa86db0efb4076f13a
Ling-Hao Chen
admin_assigned
2024-05-02T09:20:17.008Z
63109a4d61cab0446e48c83b
https://cdn-avatars.huggingface.co/v1/production/uploads/63109a4d61cab0446e48c83b/JQlVkQp0ok586ND1GmB0w.png
Ling-Hao Chen
user
EvanTHU
6631c8aa86db0efb4076f13b
Jingbo Wang
admin_assigned
2024-05-02T08:14:39.043Z
64475e73d0a1fa3c51434692
/avatars/c5cbaf118827de647a57fdaaa4a77c9e.svg
Wang
user
Jingbo520
6631c8aa86db0efb4076f13c
Jinpeng Liu
admin_assigned
2024-05-02T08:14:47.023Z
64ed85bfb6d4b4b02a9174f9
/avatars/dff01f2333f27634b18ff9a9d293c516.svg
jinpengliu
user
jinpengliu
6631c8aa86db0efb4076f13d
Bo Dai
6631c8aa86db0efb4076f13e
Yansong Tang
MotionLCM: Real-time Controllable Motion Generation via Latent
  Consistency Model
This work introduces MotionLCM, extending controllable motion generation to a
real-time level. Existing methods for spatial control in text-conditioned
motion generation suffer from significant runtime inefficiency. To address this
issue, we first propose the motion latent consistency model (MotionLCM) for
motion generation, building upon the latent diffusion model (MLD). By employing
one-step (or few-step) inference, we further improve the runtime efficiency of
the motion latent diffusion model for motion generation. To ensure effective
controllability, we incorporate a motion ControlNet within the latent space of
MotionLCM and enable explicit control signals (e.g., pelvis trajectory) in the
vanilla motion space to control the generation process directly, similar to
controlling other latent-free diffusion models for motion generation. By
employing these techniques, our approach can generate human motions with text
and control signals in real-time. Experimental results demonstrate the
remarkable generation and controlling capabilities of MotionLCM while
maintaining real-time runtime efficiency.
MotionLCM generates real-time controllable human motions using text and control signals by extending latent diffusion models with a Motion ControlNet.
motion latent consistency model MotionLCM latent diffusion model Motion ControlNet real-time motion generation motion control signals human motions