title: MotionCLR: Motion Generation and Training-free Editing via Understanding
    Attention Mechanisms
summary: This research delves into the problem of interactive editing of human motion
  generation. Previous motion diffusion models lack explicit modeling of the
  word-level text-motion correspondence and good explainability, hence
  restricting their fine-grained editing ability. To address this issue, we
  propose an attention-based motion diffusion model, namely MotionCLR, with CLeaR
  modeling of attention mechanisms. Technically, MotionCLR models the in-modality
  and cross-modality interactions with self-attention and cross-attention,
  respectively. More specifically, the self-attention mechanism aims to measure
  the sequential similarity between frames and impacts the order of motion
  features. By contrast, the cross-attention mechanism works to find the
  fine-grained word-sequence correspondence and activate the corresponding
  timesteps in the motion sequence. Based on these key properties, we develop a
  versatile set of simple yet effective motion editing methods via manipulating
  attention maps, such as motion (de-)emphasizing, in-place motion replacement,
  and example-based motion generation, etc. For further verification of the
  explainability of the attention mechanism, we additionally explore the
  potential of action-counting and grounded motion generation ability via
  attention maps. Our experimental results show that our method enjoys good
  generation and editing ability with good explainability.
paper_authors._id: 671b2439bc2e7f71a43de086
paper_authors.name: Ling-Hao Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:04:17.380Z
paper_authors._id: 63109a4d61cab0446e48c83b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63109a4d61cab0446e48c83b/JQlVkQp0ok586ND1GmB0w.png
paper_authors.fullname: Ling-Hao Chen
paper_authors.type: user
paper_authors.user: EvanTHU
paper_authors._id: 671b2439bc2e7f71a43de087
paper_authors.name: Wenxun Dai
paper_authors._id: 671b2439bc2e7f71a43de088
paper_authors.name: Xuan Ju
paper_authors._id: 671b2439bc2e7f71a43de089
paper_authors.name: Shunlin Lu
paper_authors._id: 671b2439bc2e7f71a43de08a
paper_authors.name: Lei Zhang
paper_title: MotionCLR: Motion Generation and Training-free Editing via Understanding
    Attention Mechanisms
paper_summary: This research delves into the problem of interactive editing of human motion
  generation. Previous motion diffusion models lack explicit modeling of the
  word-level text-motion correspondence and good explainability, hence
  restricting their fine-grained editing ability. To address this issue, we
  propose an attention-based motion diffusion model, namely MotionCLR, with CLeaR
  modeling of attention mechanisms. Technically, MotionCLR models the in-modality
  and cross-modality interactions with self-attention and cross-attention,
  respectively. More specifically, the self-attention mechanism aims to measure
  the sequential similarity between frames and impacts the order of motion
  features. By contrast, the cross-attention mechanism works to find the
  fine-grained word-sequence correspondence and activate the corresponding
  timesteps in the motion sequence. Based on these key properties, we develop a
  versatile set of simple yet effective motion editing methods via manipulating
  attention maps, such as motion (de-)emphasizing, in-place motion replacement,
  and example-based motion generation, etc. For further verification of the
  explainability of the attention mechanism, we additionally explore the
  potential of action-counting and grounded motion generation ability via
  attention maps. Our experimental results show that our method enjoys good
  generation and editing ability with good explainability.
paper_ai_summary: MotionCLR, an attention-based motion diffusion model, enhances interactive motion editing by explicitly modeling word-level text-motion correspondence and improving explainability through self-attention and cross-attention mechanisms.
paper_ai_keywords: attention-based motion diffusion model MotionCLR CLeaR modeling self-attention cross-attention text-motion correspondence motion editing explainability attention maps motion (de-)emphasizing in-place motion replacement example-based motion generation action-counting grounded motion generation