title: MixerMDM: Learnable Composition of Human Motion Diffusion Models
summary: Generating human motion guided by conditions such as textual descriptions is
  challenging due to the need for datasets with pairs of high-quality motion and
  their corresponding conditions. The difficulty increases when aiming for finer
  control in the generation. To that end, prior works have proposed to combine
  several motion diffusion models pre-trained on datasets with different types of
  conditions, thus allowing control with multiple conditions. However, the
  proposed merging strategies overlook that the optimal way to combine the
  generation processes might depend on the particularities of each pre-trained
  generative model and also the specific textual descriptions. In this context,
  we introduce MixerMDM, the first learnable model composition technique for
  combining pre-trained text-conditioned human motion diffusion models. Unlike
  previous approaches, MixerMDM provides a dynamic mixing strategy that is
  trained in an adversarial fashion to learn to combine the denoising process of
  each model depending on the set of conditions driving the generation. By using
  MixerMDM to combine single- and multi-person motion diffusion models, we
  achieve fine-grained control on the dynamics of every person individually, and
  also on the overall interaction. Furthermore, we propose a new evaluation
  technique that, for the first time in this task, measures the interaction and
  individual quality by computing the alignment between the mixed generated
  motions and their conditions as well as the capabilities of MixerMDM to adapt
  the mixing throughout the denoising process depending on the motions to mix.
paper_authors._id: 67ecd893553eb1c01a311882
paper_authors.name: Pablo Ruiz-Ponce
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:21:16.654Z
paper_authors._id: 62ace9bc717ee4c12b72e275
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62ace9bc717ee4c12b72e275/VfR-7R3H-5FV0OILa0XzL.jpeg
paper_authors.fullname: Pablo Ruiz-Ponce
paper_authors.type: user
paper_authors.user: pabloruizponce
paper_authors._id: 67ecd893553eb1c01a311883
paper_authors.name: German Barquero
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:53:49.237Z
paper_authors._id: 64fac0f3b961d0d12c756b59
paper_authors.avatarUrl: /avatars/27cc7dbad927df818ba2f91a5b6942f9.svg
paper_authors.fullname: German Barquero
paper_authors.type: user
paper_authors.user: Germs96
paper_authors._id: 67ecd893553eb1c01a311884
paper_authors.name: Cristina Palmero
paper_authors._id: 67ecd893553eb1c01a311885
paper_authors.name: Sergio Escalera
paper_authors._id: 67ecd893553eb1c01a311886
paper_authors.name: José García-Rodríguez
paper_title: MixerMDM: Learnable Composition of Human Motion Diffusion Models
paper_summary: Generating human motion guided by conditions such as textual descriptions is
  challenging due to the need for datasets with pairs of high-quality motion and
  their corresponding conditions. The difficulty increases when aiming for finer
  control in the generation. To that end, prior works have proposed to combine
  several motion diffusion models pre-trained on datasets with different types of
  conditions, thus allowing control with multiple conditions. However, the
  proposed merging strategies overlook that the optimal way to combine the
  generation processes might depend on the particularities of each pre-trained
  generative model and also the specific textual descriptions. In this context,
  we introduce MixerMDM, the first learnable model composition technique for
  combining pre-trained text-conditioned human motion diffusion models. Unlike
  previous approaches, MixerMDM provides a dynamic mixing strategy that is
  trained in an adversarial fashion to learn to combine the denoising process of
  each model depending on the set of conditions driving the generation. By using
  MixerMDM to combine single- and multi-person motion diffusion models, we
  achieve fine-grained control on the dynamics of every person individually, and
  also on the overall interaction. Furthermore, we propose a new evaluation
  technique that, for the first time in this task, measures the interaction and
  individual quality by computing the alignment between the mixed generated
  motions and their conditions as well as the capabilities of MixerMDM to adapt
  the mixing throughout the denoising process depending on the motions to mix.
paper_ai_summary: MixerMDM, a learnable technique, dynamically combines pre-trained text-conditioned motion diffusion models to achieve fine-grained control over individual and interactive human motions, using an adversarial training approach and new evaluation metrics.
paper_ai_keywords: motion diffusion models text-conditioned learnable model composition dynamic mixing adversarial fashion denoising process