<p>One encoder block of the transformer takes as input one tensor X and multiplies that by <span class="math-container">$W_Q$</span>, <span class="math-container">$W_K$</span>, <span class="math-container">$W_V$</span> to calculate <span class="math-container">$Q$</span>, <span class="math-container">$K$</span>, <span class="math-container">$V$</span> needed in self-attention.</p>
<p>After performing attention and feed-forward this one encoder block returns a single <span class="math-container">$X'$</span> ready to be taken as input for the next encoder block.</p>
<p>I find this specific post really helpful:</p>
<p><a href="https://jalammar.github.io/illustrated-transformer/" rel="nofollow noreferrer">https://jalammar.github.io/illustrated-transformer/</a></p>
<p>You can find there this image, which shows that from one single input X, we calculate <span class="math-container">$Q, K, V$</span>. Learning those weight matrices <span class="math-container">$W_Q$</span>, <span class="math-container">$W_K$</span>, <span class="math-container">$W_V$</span> is part of the training of an encoder.
<a href="https://i.sstatic.net/s6kuJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/s6kuJ.png" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2023-02-28T22:32:43.787Z
2023-02-28T22:32:43.787Z
Jakub Podolak
<p>An Encoder has as inputs : Q,K,V, but has single output i.e. 3 vs 1</p>
<p>How do you stack those ?</p>
<p>Is there more detailed diagram ?</p>

CC BY-SA 4.0
2023-02-28T10:26:37.133Z
Pass the output through 3 different dense layers in parallel, I think.
Criticize SE actions means ban
<transformer><encoder-decoder>
Transformers: how does stacking work?
sten
# Transformers: how does stacking work?

(Asked by: username_0 on 2023-02-28)

An Encoder has as inputs : Q,K,V, but has single output i.e. 3 vs 1

How do you stack those ?

Is there more detailed diagram ?

**Tags:** transformer, encoder-decoder

> **Comments:**
> **username_1**: Pass the output through 3 different dense layers in parallel, I think.  

---

## 1 answers

### Answer by username_2 on 2023-02-28. Score: 2 | Accepted answer

One encoder block of the transformer takes as input one tensor X and multiplies that by $W\_Q$, $W\_K$, $W\_V$ to calculate $Q$, $K$, $V$ needed in self-attention.

After performing attention and feed-forward this one encoder block returns a single $X'$ ready to be taken as input for the next encoder block.

I find this specific post really helpful:

<https://jalammar.github.io/illustrated-transformer/>

You can find there this image, which shows that from one single input X, we calculate $Q, K, V$. Learning those weight matrices $W\_Q$, $W\_K$, $W\_V$ is part of the training of an encoder.
[![enter image description here](https://i.sstatic.net/s6kuJ.png)](https://i.sstatic.net/s6kuJ.png)


---

