title: MoMask: Generative Masked Modeling of 3D Human Motions
summary: We introduce MoMask, a novel masked modeling framework for text-driven 3D
  human motion generation. In MoMask, a hierarchical quantization scheme is
  employed to represent human motion as multi-layer discrete motion tokens with
  high-fidelity details. Starting at the base layer, with a sequence of motion
  tokens obtained by vector quantization, the residual tokens of increasing
  orders are derived and stored at the subsequent layers of the hierarchy. This
  is consequently followed by two distinct bidirectional transformers. For the
  base-layer motion tokens, a Masked Transformer is designated to predict
  randomly masked motion tokens conditioned on text input at training stage.
  During generation (i.e. inference) stage, starting from an empty sequence, our
  Masked Transformer iteratively fills up the missing tokens; Subsequently, a
  Residual Transformer learns to progressively predict the next-layer tokens
  based on the results from current layer. Extensive experiments demonstrate that
  MoMask outperforms the state-of-art methods on the text-to-motion generation
  task, with an FID of 0.045 (vs e.g. 0.141 of T2M-GPT) on the HumanML3D dataset,
  and 0.228 (vs 0.514) on KIT-ML, respectively. MoMask can also be seamlessly
  applied in related tasks without further model fine-tuning, such as text-guided
  temporal inpainting.
paper_authors._id: 656d3e0e4ab7bc884dbe6d47
paper_authors.name: Chuan Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T20:51:33.037Z
paper_authors._id: 656d77a08a37acfa3f275d46
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656d77a08a37acfa3f275d46/3S_TakMfG5J8oZ6qDs-SX.jpeg
paper_authors.fullname: chuan guo
paper_authors.type: user
paper_authors.user: Ericguo5513
paper_authors._id: 656d3e0e4ab7bc884dbe6d48
paper_authors.name: Yuxuan Mu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:40:51.493Z
paper_authors._id: 64ac2cce5769f5f88bc25475
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ac2cce5769f5f88bc25475/SMpNfbmDQd7IRIF0WBttV.jpeg
paper_authors.fullname: Mu
paper_authors.type: user
paper_authors.user: MeYourHint
paper_authors._id: 656d3e0e4ab7bc884dbe6d49
paper_authors.name: Muhammad Gohar Javed
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:40:47.071Z
paper_authors._id: 656e260e801ed9952ffee88b
paper_authors.avatarUrl: /avatars/d9032cf3fed127d1234953af6b551fa8.svg
paper_authors.fullname: Muhammad Gohar Javed
paper_authors.type: user
paper_authors.user: javed4
paper_authors._id: 656d3e0e4ab7bc884dbe6d4a
paper_authors.name: Sen Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:40:43.595Z
paper_authors._id: 64384682581e6bf0fa9ce6f8
paper_authors.avatarUrl: /avatars/b4b5b4ed8b34b8d225f27c13d99985cd.svg
paper_authors.fullname: Sen Wang
paper_authors.type: user
paper_authors.user: wangsenxxz
paper_authors._id: 656d3e0e4ab7bc884dbe6d4b
paper_authors.name: Li Cheng
paper_title: MoMask: Generative Masked Modeling of 3D Human Motions
paper_summary: We introduce MoMask, a novel masked modeling framework for text-driven 3D
  human motion generation. In MoMask, a hierarchical quantization scheme is
  employed to represent human motion as multi-layer discrete motion tokens with
  high-fidelity details. Starting at the base layer, with a sequence of motion
  tokens obtained by vector quantization, the residual tokens of increasing
  orders are derived and stored at the subsequent layers of the hierarchy. This
  is consequently followed by two distinct bidirectional transformers. For the
  base-layer motion tokens, a Masked Transformer is designated to predict
  randomly masked motion tokens conditioned on text input at training stage.
  During generation (i.e. inference) stage, starting from an empty sequence, our
  Masked Transformer iteratively fills up the missing tokens; Subsequently, a
  Residual Transformer learns to progressively predict the next-layer tokens
  based on the results from current layer. Extensive experiments demonstrate that
  MoMask outperforms the state-of-art methods on the text-to-motion generation
  task, with an FID of 0.045 (vs e.g. 0.141 of T2M-GPT) on the HumanML3D dataset,
  and 0.228 (vs 0.514) on KIT-ML, respectively. MoMask can also be seamlessly
  applied in related tasks without further model fine-tuning, such as text-guided
  temporal inpainting.
paper_ai_keywords: MoMask masked modeling text-driven 3D human motion generation hierarchical quantization discrete motion tokens vector quantization residual tokens bidirectional transformers Masked Transformer Residual Transformer text-to-motion generation FID HumanML3D dataset KIT-ML text-guided temporal inpainting