<p>in the original paper the following scheme of the self-attention appears:
<a href="https://arxiv.org/pdf/1805.08318.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1805.08318.pdf</a>
<a href="https://i.sstatic.net/jkicQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jkicQ.png" alt="enter image description here" /></a></p>
<p>In a later overview:
<a href="https://arxiv.org/pdf/1906.01529.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1906.01529.pdf</a></p>
<p>this scheme appears:
<a href="https://i.sstatic.net/9u9a0.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9u9a0.png" alt="enter image description here" /></a>
referring the original paper.</p>
<p>My understanding more correlates with the second paper scheme, as:
<a href="https://i.sstatic.net/H8uJ4.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/H8uJ4.png" alt="enter image description here" /></a>
Where there is two dot-product operations and three hidden parametric matrices:
<span class="math-container">$$W_k, W_v, W_q$$</span>
which corresponds to <span class="math-container">$W_f, W_g, W_h$</span> without <span class="math-container">$W_v$</span> as it in the original paper explanation, which is as following:</p>
<p><a href="https://i.sstatic.net/QwCDY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QwCDY.png" alt="enter image description here" /></a></p>
<p>Is this a mistake in the original paper ?</p>

CC BY-SA 4.0
2021-03-29T10:36:40.217Z
I don't understand where the mistake is. Can you please clarify where the mistake is?
nbro
CC BY-SA 4.0
2021-03-29T14:10:11.813Z
I think I was confused by the notation of v as a vector function, I thought it is a vector and then it didn't work out. So eventually there is non.
Ilya.K.
<deep-neural-networks><generative-adversarial-networks><image-processing><attention><attn-gan>
SAGAN - is there a mistake in the original paper?
Ilya.K.
# SAGAN - is there a mistake in the original paper?

(Asked by: username_0 on 2021-03-28)

in the original paper the following scheme of the self-attention appears:
<https://arxiv.org/pdf/1805.08318.pdf>
[![enter image description here](https://i.sstatic.net/jkicQ.png)](https://i.sstatic.net/jkicQ.png)

In a later overview:
<https://arxiv.org/pdf/1906.01529.pdf>

this scheme appears:
[![enter image description here](https://i.sstatic.net/9u9a0.png)](https://i.sstatic.net/9u9a0.png)
referring the original paper.

My understanding more correlates with the second paper scheme, as:
[![enter image description here](https://i.sstatic.net/H8uJ4.png)](https://i.sstatic.net/H8uJ4.png)
Where there is two dot-product operations and three hidden parametric matrices:
$$W\_k, W\_v, W\_q$$
which corresponds to $W\_f, W\_g, W\_h$ without $W\_v$ as it in the original paper explanation, which is as following:

[![enter image description here](https://i.sstatic.net/QwCDY.png)](https://i.sstatic.net/QwCDY.png)

Is this a mistake in the original paper ?

**Tags:** deep-neural-networks, generative-adversarial-networks, image-processing, attention, attn-gan

> **Comments:**
> **username_1**: I don't understand where the mistake is. Can you please clarify where the mistake is?  
> **username_0**: I think I was confused by the notation of v as a vector function, I thought it is a vector and then it didn't work out. So eventually there is non.  

---

## 0 answers

