Body: <p>in the original paper the following scheme of the self-attention appears:
  <a href="https://arxiv.org/pdf/1805.08318.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1805.08318.pdf</a>
  <a href="https://i.sstatic.net/jkicQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jkicQ.png" alt="enter image description here" /></a></p>
  <p>In a later overview:
  <a href="https://arxiv.org/pdf/1906.01529.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1906.01529.pdf</a></p>
  <p>this scheme appears:
  <a href="https://i.sstatic.net/9u9a0.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9u9a0.png" alt="enter image description here" /></a>
  referring the original paper.</p>
  <p>My understanding more correlates with the second paper scheme, as:
  <a href="https://i.sstatic.net/H8uJ4.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/H8uJ4.png" alt="enter image description here" /></a>
  Where there is two dot-product operations and three hidden parametric matrices:
  <span class="math-container">$$W_k, W_v, W_q$$</span>
  which corresponds to <span class="math-container">$W_f, W_g, W_h$</span> without <span class="math-container">$W_v$</span> as it in the original paper explanation, which is as following:</p>
  <p><a href="https://i.sstatic.net/QwCDY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QwCDY.png" alt="enter image description here" /></a></p>
  <p>Is this a mistake in the original paper ?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-29T10:36:40.217Z
Comments.Text: I don't understand where the mistake is. Can you please clarify where the mistake is?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-29T14:10:11.813Z
Comments.Text: I think I was confused by the notation of v as a vector function, I thought it is a vector and then it didn't work out. So eventually there is non.
Comments.UserDisplayName: Ilya.K.
Tags: <deep-neural-networks><generative-adversarial-networks><image-processing><attention><attn-gan>
Title: SAGAN - is there a mistake in the original paper?
UserDisplayName: Ilya.K.
ThreadText: # SAGAN - is there a mistake in the original paper?
  
  (Asked by: username_0 on 2021-03-28)
  
  in the original paper the following scheme of the self-attention appears:
  <https://arxiv.org/pdf/1805.08318.pdf>
  [![enter image description here](https://i.sstatic.net/jkicQ.png)](https://i.sstatic.net/jkicQ.png)
  
  In a later overview:
  <https://arxiv.org/pdf/1906.01529.pdf>
  
  this scheme appears:
  [![enter image description here](https://i.sstatic.net/9u9a0.png)](https://i.sstatic.net/9u9a0.png)
  referring the original paper.
  
  My understanding more correlates with the second paper scheme, as:
  [![enter image description here](https://i.sstatic.net/H8uJ4.png)](https://i.sstatic.net/H8uJ4.png)
  Where there is two dot-product operations and three hidden parametric matrices:
  $$W\_k, W\_v, W\_q$$
  which corresponds to $W\_f, W\_g, W\_h$ without $W\_v$ as it in the original paper explanation, which is as following:
  
  [![enter image description here](https://i.sstatic.net/QwCDY.png)](https://i.sstatic.net/QwCDY.png)
  
  Is this a mistake in the original paper ?
  
  **Tags:** deep-neural-networks, generative-adversarial-networks, image-processing, attention, attn-gan
  > **Comments:**
  > **username_1**: I don't understand where the mistake is. Can you please clarify where the mistake is?  
  > **username_0**: I think I was confused by the notation of v as a vector function, I thought it is a vector and then it didn't work out. So eventually there is non.  
  
  ---
  
  ## 0 answers
  
  