<p>The self-attention part 'enriches' the motion sequence. The cross-attention module mixes two different embedding sequences. In a language translation model, the cross-attention module is the part where the actual translation happens. In the MotionDiffuse paper, this is the part where the two modalities (motion sequence and text) interact.</p>
<p><a href="https://i.sstatic.net/h6hjs.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/h6hjs.png" alt="minimax game" /></a></p>
<p>The original attention mechanism however does not scale very well. Computing the attention weights leads to quadratically increasing costs. The MotionDiffuse paper makes use of the speed-ups introduced in the</p>
<blockquote>
<p>Efficient Attention paper (Shen et al., 2021). Instead of calculating pair-wise attention weights, efficient attention generates global
feature map.</p>
</blockquote>
<p>See this <a href="https://www.youtube.com/watch?v=EixI6t5oif0" rel="nofollow noreferrer">video</a> for a detailed explanation of the original transformer architecture.</p>

CC BY-SA 4.0
2023-01-02T22:26:56.763Z
Thanks, could this model be considered a latent diffusion model? I just finished reading [this](https://theaisummer.com/diffusion-models/#stable-diffusion-latent-diffusion-models) and it seemed to fit the description.
Nikita Belooussov
CC BY-SA 4.0
2023-01-02T21:24:56.293Z
2023-01-02T21:24:56.293Z
Mariusmarten
<p>So I am trying to understand how a Cross-Modality Linear Transformer is different from an a basic transformer. I found the transformer mentioned in this <a href="https://arxiv.org/pdf/2208.15001.pdf" rel="nofollow noreferrer">paper</a>. Am I correct in understanding that, the transformer  is cross-modality because it is going from one medium to another. In this case, text to motion. Then linear because the time complexity to calculate the attetion weights are linear? I have troubles with understanding transformers. I just was able to understand the basic concept of them. So any help would be great. Thanks</p>

<transformer>
Explanation of Cross-Modality Linear Transformer
Nikita Belooussov
# Explanation of Cross-Modality Linear Transformer

(Asked by: username_0 on 2022-12-31)

So I am trying to understand how a Cross-Modality Linear Transformer is different from an a basic transformer. I found the transformer mentioned in this [paper](https://arxiv.org/pdf/2208.15001.pdf). Am I correct in understanding that, the transformer is cross-modality because it is going from one medium to another. In this case, text to motion. Then linear because the time complexity to calculate the attetion weights are linear? I have troubles with understanding transformers. I just was able to understand the basic concept of them. So any help would be great. Thanks

**Tags:** transformer


---

## 1 answers

### Answer by username_1 on 2023-01-02. Score: 1 | Accepted answer

The self-attention part 'enriches' the motion sequence. The cross-attention module mixes two different embedding sequences. In a language translation model, the cross-attention module is the part where the actual translation happens. In the MotionDiffuse paper, this is the part where the two modalities (motion sequence and text) interact.

[![minimax game](https://i.sstatic.net/h6hjs.png)](https://i.sstatic.net/h6hjs.png)

The original attention mechanism however does not scale very well. Computing the attention weights leads to quadratically increasing costs. The MotionDiffuse paper makes use of the speed-ups introduced in the

> Efficient Attention paper (Shen et al., 2021). Instead of calculating pair-wise attention weights, efficient attention generates global
> feature map.

See this [video](https://www.youtube.com/watch?v=EixI6t5oif0) for a detailed explanation of the original transformer architecture.

> **Comments:**
> **username_0**: Thanks, could this model be considered a latent diffusion model? I just finished reading [this](https://theaisummer.com/diffusion-models/#stable-diffusion-latent-diffusion-models) and it seemed to fit the description.  

---

