ReMoMask: Retrieval-Augmented Masked Motion Generation
Text-to-Motion (T2M) generation aims to synthesize realistic and semantically
aligned human motion sequences from natural language descriptions. However,
current approaches face dual challenges: Generative models (e.g., diffusion
models) suffer from limited diversity, error accumulation, and physical
implausibility, while Retrieval-Augmented Generation (RAG) methods exhibit
diffusion inertia, partial-mode collapse, and asynchronous artifacts. To
address these limitations, we propose ReMoMask, a unified framework integrating
three key innovations: 1) A Bidirectional Momentum Text-Motion Model decouples
negative sample scale from batch size via momentum queues, substantially
improving cross-modal retrieval precision; 2) A Semantic Spatio-temporal
Attention mechanism enforces biomechanical constraints during part-level fusion
to eliminate asynchronous artifacts; 3) RAG-Classier-Free Guidance incorporates
minor unconditional generation to enhance generalization. Built upon MoMask's
RVQ-VAE, ReMoMask efficiently generates temporally coherent motions in minimal
steps. Extensive experiments on standard benchmarks demonstrate the
state-of-the-art performance of ReMoMask, achieving a 3.88% and 10.97%
improvement in FID scores on HumanML3D and KIT-ML, respectively, compared to
the previous SOTA method RAG-T2M. Code:
https://github.com/AIGeeksGroup/ReMoMask. Website:
https://aigeeksgroup.github.io/ReMoMask.
6891de219373cd501b6a2d32
Zhengdao Li
6891de219373cd501b6a2d33
Siheng Wang
6891de219373cd501b6a2d34
Zeyu Zhang
6891de219373cd501b6a2d35
Hao Tang
ReMoMask: Retrieval-Augmented Masked Motion Generation
Text-to-Motion (T2M) generation aims to synthesize realistic and semantically
aligned human motion sequences from natural language descriptions. However,
current approaches face dual challenges: Generative models (e.g., diffusion
models) suffer from limited diversity, error accumulation, and physical
implausibility, while Retrieval-Augmented Generation (RAG) methods exhibit
diffusion inertia, partial-mode collapse, and asynchronous artifacts. To
address these limitations, we propose ReMoMask, a unified framework integrating
three key innovations: 1) A Bidirectional Momentum Text-Motion Model decouples
negative sample scale from batch size via momentum queues, substantially
improving cross-modal retrieval precision; 2) A Semantic Spatio-temporal
Attention mechanism enforces biomechanical constraints during part-level fusion
to eliminate asynchronous artifacts; 3) RAG-Classier-Free Guidance incorporates
minor unconditional generation to enhance generalization. Built upon MoMask's
RVQ-VAE, ReMoMask efficiently generates temporally coherent motions in minimal
steps. Extensive experiments on standard benchmarks demonstrate the
state-of-the-art performance of ReMoMask, achieving a 3.88% and 10.97%
improvement in FID scores on HumanML3D and KIT-ML, respectively, compared to
the previous SOTA method RAG-T2M. Code:
https://github.com/AIGeeksGroup/ReMoMask. Website:
https://aigeeksgroup.github.io/ReMoMask.
ReMoMask, a unified framework, addresses limitations in text-to-motion generation by integrating a Bidirectional Momentum Text-Motion Model, Semantic Spatio-temporal Attention, and RAG-Classier-Free Guidance, achieving state-of-the-art performance on HumanML3D and KIT-ML benchmarks.
diffusion models Retrieval-Augmented Generation (RAG) Bidirectional Momentum Text-Motion Model momentum queues Semantic Spatio-temporal Attention biomechanical constraints RAG-Classier-Free Guidance RVQ-VAE temporally coherent motions FID scores