Answers.Body: <p>I guess this picture is not quite right: &quot;And then these [attention weights?] can be 'fed' to the decoder part of the network which will also consider word vectors that have been produced by the decoder so far and influence word selection by paying special attention to particular word combinations.&quot;</p>
  <p>The general picture I have learned goes like this:</p>
  <ol>
  <li><p>One-hot word vectors are sent through the embedding matrix <span class="math-container">$W$</span> and yield lower-dimensional word embeddings.</p>
  </li>
  <li><p>In each decoder layer the word vectors are enriched by the attention mechanism, but only by contextual information from previous tokens (&quot;masked attention&quot;).</p>
  </li>
  <li><p>So the first token doesn't get enriched at all, its information content remains the same.</p>
  </li>
  <li><p>The last token takes up the maximum of information available and after the last decoder represents the context as a whole.</p>
  </li>
  <li><p>It's only the enriched word embedding (= context embedding) of the last token that is now sent through the transpose <span class="math-container">$W^T$</span> of the word embedding matrix (the final decoder, different from the transformer decoders). It yields the logits vector.</p>
  </li>
  </ol>
  <p>Other models than GPT may use other final decoders than <span class="math-container">$W^T$</span>.</p>
  <p>The following diagram depicts this. The shades of gray of the word vectors passed through the transformer decoders display their informational content (or density).</p>
  <p><a href="https://i.sstatic.net/1HBTJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1HBTJ.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-16T11:45:23.220Z
Answers.LastActivityDate: 2023-06-16T11:45:23.220Z
Answers.UserDisplayName: Hans-Peter Stricker
Answers.Body: <p>After forwarding the prompt through the whole <a href="https://ai.stackexchange.com/questions/40179/how-does-the-decoder-only-transformer-architecture-work">transformer architecture</a>, the output is forwarded through one final linear layer.</p>
  <blockquote>
  <p>&quot;This final linear layer maps the output of the model back to the size of the 'vocabulary'. I.e. if you want to predict the next letter in a message, it would map to 26 (letters) + additional stuff (such as .,-!? etc.).&quot;</p>
  </blockquote>
  <p>For each possible output token, it predicts the probability (between 0 and 1) that that token is the next token. For models such as GPT-4, this last linear layer maps to a 10.000+ dimensional output (it includes arabian/chinese/japanese etc tokens).</p>
  <p>Regarding your question:</p>
  <blockquote>
  <p>Is it the case that the decoder &quot;considers&quot; each possible word in it's entire vocabulary when producing an output word?</p>
  </blockquote>
  <p>Yes. Each output token is considered as a possible option.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-14T15:32:29.647Z
Answers.LastActivityDate: 2023-05-14T15:32:29.647Z
Answers.UserDisplayName: Robin van Hoorn
Body: <p>I have a very rough understanding of the &quot;attention/self attention&quot; mechanism of transformer models and how this can be used to process a set of word vectors provided as an input/prompt to the encoder of a network and how this will produce &quot;attention weights&quot; for the word vectors based on positional encodings and some other learnable parameters (key/query/value transforms). And then these can be &quot;fed&quot; to the decoder part of the network which will also consider word vectors that have been produced by the decoder so far and influence word selection by paying special attention to particular word combinations.</p>
  <p>However LLMs clearly produce words in their output/response that do not occur anywhere inside the &quot;prompt&quot; text. So they must be using these &quot;attention weights&quot; to consider words from a wider vocabulary, which could be quite large.</p>
  <p>Is it the case that the decoder &quot;considers&quot; each possible word in it's entire vocabulary when producing an output word? For example I'm imagining an input layer to a NN with several thousand nodes (one per word vector in dictionary) on the input then these are &quot;combined&quot; through some operation with attention weights (from the encoder and decoder &quot;attention section&quot;) producing values for most word vectors that are very low (so below the threshold for some activation function) but each word is still &quot;considered&quot; to an extent? Or are only a subset of words considered in some way?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-16T13:39:27.747Z
Comments.Text: All LLMs have a fixed vocabulary of tokens. In the simplest case, a token is a word. This means words not in the vocabulary cannot be output. Modern LLMs have a tokeniser that converts text into tokens. The raw output of the LLM is in tokens. These are strung together to form sentences. There is some additional work to convert a distribution over tokens into a predicted sentence. The simplest model is to choose the token/word with the highest probability in each position. That may yield nonsensical outputs. Real-world LLMs do some clever searching and post processing.
Comments.UserDisplayName: Anirban Mukherjee
Tags: <neural-networks><natural-language-processing><transformer><natural-language-generation><large-language-models>
Title: How does a LLM (transformer) pick words from its vocabulary?
UserDisplayName: MLBeginner
ThreadText: # How does a LLM (transformer) pick words from its vocabulary?
  
  (Asked by: username_0 on 2023-03-28)
  
  I have a very rough understanding of the "attention/self attention" mechanism of transformer models and how this can be used to process a set of word vectors provided as an input/prompt to the encoder of a network and how this will produce "attention weights" for the word vectors based on positional encodings and some other learnable parameters (key/query/value transforms). And then these can be "fed" to the decoder part of the network which will also consider word vectors that have been produced by the decoder so far and influence word selection by paying special attention to particular word combinations.
  
  However LLMs clearly produce words in their output/response that do not occur anywhere inside the "prompt" text. So they must be using these "attention weights" to consider words from a wider vocabulary, which could be quite large.
  
  Is it the case that the decoder "considers" each possible word in it's entire vocabulary when producing an output word? For example I'm imagining an input layer to a NN with several thousand nodes (one per word vector in dictionary) on the input then these are "combined" through some operation with attention weights (from the encoder and decoder "attention section") producing values for most word vectors that are very low (so below the threshold for some activation function) but each word is still "considered" to an extent? Or are only a subset of words considered in some way?
  
  **Tags:** neural-networks, natural-language-processing, transformer, natural-language-generation, large-language-models
  
  > **Comments:**
  > **username_1**: All LLMs have a fixed vocabulary of tokens. In the simplest case, a token is a word. This means words not in the vocabulary cannot be output. Modern LLMs have a tokeniser that converts text into tokens. The raw output of the LLM is in tokens. These are strung together to form sentences. There is some additional work to convert a distribution over tokens into a predicted sentence. The simplest model is to choose the token/word with the highest probability in each position. That may yield nonsensical outputs. Real-world LLMs do some clever searching and post processing.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-06-16. Score: 1
  
  I guess this picture is not quite right: "And then these [attention weights?] can be 'fed' to the decoder part of the network which will also consider word vectors that have been produced by the decoder so far and influence word selection by paying special attention to particular word combinations."
  
  The general picture I have learned goes like this:
  
  1. One-hot word vectors are sent through the embedding matrix $W$ and yield lower-dimensional word embeddings.
  2. In each decoder layer the word vectors are enriched by the attention mechanism, but only by contextual information from previous tokens ("masked attention").
  3. So the first token doesn't get enriched at all, its information content remains the same.
  4. The last token takes up the maximum of information available and after the last decoder represents the context as a whole.
  5. It's only the enriched word embedding (= context embedding) of the last token that is now sent through the transpose $W^T$ of the word embedding matrix (the final decoder, different from the transformer decoders). It yields the logits vector.
  
  Other models than GPT may use other final decoders than $W^T$.
  
  The following diagram depicts this. The shades of gray of the word vectors passed through the transformer decoders display their informational content (or density).
  
  [![enter image description here](https://i.sstatic.net/1HBTJ.png)](https://i.sstatic.net/1HBTJ.png)
  
  
  ---
  
  ### Answer by username_3 on 2023-05-14. Score: 0
  
  After forwarding the prompt through the whole [transformer architecture](https://ai.stackexchange.com/questions/40179/how-does-the-decoder-only-transformer-architecture-work), the output is forwarded through one final linear layer.
  
  > "This final linear layer maps the output of the model back to the size of the 'vocabulary'. I.e. if you want to predict the next letter in a message, it would map to 26 (letters) + additional stuff (such as .,-!? etc.)."
  
  For each possible output token, it predicts the probability (between 0 and 1) that that token is the next token. For models such as GPT-4, this last linear layer maps to a 10.000+ dimensional output (it includes arabian/chinese/japanese etc tokens).
  
  Regarding your question:
  
  > Is it the case that the decoder "considers" each possible word in it's entire vocabulary when producing an output word?
  
  Yes. Each output token is considered as a possible option.
  
  
  ---
  
  