<p>(This type of) autoregressive LLM always works by predicting <em>one</em> next token based on a series of previous tokens. First you run the model with input &quot;today is a&quot; and the prediction is &quot;good&quot;. Then you run the model with input &quot;today is a good&quot; and the prediction is &quot;day&quot;, and so on. Each token is predicted by running the entire model from start to finish on its previous input.</p>

CC BY-SA 4.0
2023-05-09T12:52:09.063Z
2023-05-09T12:52:09.063Z
Criticize SE actions means ban
<p>For a transformer decoder, how exactly are K, Q, and V for each decoding step?</p>
<p>Assume my input prompt is &quot;today is a&quot; (good day).</p>
<p>At t= 0 (generation step 0):
K, Q, and V are the projections of the sequence (&quot;today is a&quot;)
Then say the next token generated is &quot;good&quot;.</p>
<p>At <code>t=1</code> (generation step 1):
Which one is true:</p>
<ol>
<li>K, Q, and V are the projections of the sequence (&quot;today is a good&quot;)</li>
<li>K, Q, are the projections of the sequence (&quot;today is a&quot;), and V is the projection of the sequence (&quot;good&quot;)?</li>
</ol>

CC BY-SA 4.0
2023-05-09T07:03:57.083Z
Please see if [the explanation](https://ai.stackexchange.com/questions/40179/how-does-the-decoder-only-transformer-architecture-work) here helps you in any way.
In short, during the forward pass, K, Q, and V are projections of the input sequence ("today is a"). The output of the forward pass is "good".
Robin van Hoorn
CC BY-SA 4.0
2023-05-10T02:33:58.030Z
@RobinvanHoorn, I see, and thanks. My question is the next next time step (t+2), so now your entire sequence is "today is a good", do you just repeat the same thing (with K, Q, V as the projections of the "today is a good" sequence?)
wrek
CC BY-SA 4.0
2023-05-10T09:50:20.783Z
your question was (and the question still only mentions) `t=1`. If you again want to predict a next word, then K,Q and V are again projections of the input sequence ("today is a good") and then the output becomes (i.e.) "day".
Robin van Hoorn
<transformer><autoencoders><large-language-models><encoder-decoder>
For a transformer decoder, how exactly are K, Q, and V for each decoding step?
wrek
# For a transformer decoder, how exactly are K, Q, and V for each decoding step?

(Asked by: username_0 on 2023-05-09)

For a transformer decoder, how exactly are K, Q, and V for each decoding step?

Assume my input prompt is "today is a" (good day).

At t= 0 (generation step 0):
K, Q, and V are the projections of the sequence ("today is a")
Then say the next token generated is "good".

At `t=1` (generation step 1):
Which one is true:

1. K, Q, and V are the projections of the sequence ("today is a good")
2. K, Q, are the projections of the sequence ("today is a"), and V is the projection of the sequence ("good")?

**Tags:** transformer, autoencoders, large-language-models, encoder-decoder

> **Comments:**
> **username_1**: Please see if [the explanation](https://ai.stackexchange.com/questions/40179/how-does-the-decoder-only-transformer-architecture-work) here helps you in any way. In short, during the forward pass, K, Q, and V are projections of the input sequence ("today is a"). The output of the forward pass is "good".  
> **username_0**: @RobinvanHoorn, I see, and thanks. My question is the next next time step (t+2), so now your entire sequence is "today is a good", do you just repeat the same thing (with K, Q, V as the projections of the "today is a good" sequence?)  
> **username_1**: your question was (and the question still only mentions) `t=1`. If you again want to predict a next word, then K,Q and V are again projections of the input sequence ("today is a good") and then the output becomes (i.e.) "day".  

---

## 1 answers

### Answer by username_2 on 2023-05-09. Score: 2 | Accepted answer

(This type of) autoregressive LLM always works by predicting *one* next token based on a series of previous tokens. First you run the model with input "today is a" and the prediction is "good". Then you run the model with input "today is a good" and the prediction is "day", and so on. Each token is predicted by running the entire model from start to finish on its previous input.


---

