Answers.Body: <p>Once the last level of the <a href="https://paperswithcode.com/method/multi-head-attention" rel="nofollow noreferrer"><em>multi-head attention</em></a> mechanism returns its output, the matrix is flattened in an array and sent as input to a <em><a href="https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear" rel="nofollow noreferrer">linear</a></em> level (a.k.a. a <em>fully-connected</em> layer).The output of this layer is activated via <a href="https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html#softmax" rel="nofollow noreferrer">Softmax</a> in order to have probabilities associated to the next token. The token with the highest probability is then returned.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-04T23:33:20.137Z
Answers.LastActivityDate: 2024-06-04T23:33:20.137Z
Answers.UserDisplayName: Nope
Body: <p>There is something that i can't get it, given a prompt input to ChatGPT, this is One Hot Encoded, Embedded, Positional Encoded and so on. Anyway we have a matrix, still after attention mechanism we have a matrix, how  we end with a probability vector and not with a probability matrix?</p>
  
Tags: <chatgpt>
Title: How ChatGPT pass from a prompt to a predicted word?
UserDisplayName: HariSeldon
ThreadText: # How ChatGPT pass from a prompt to a predicted word?
  
  (Asked by: username_0 on 2024-06-04)
  
  There is something that i can't get it, given a prompt input to ChatGPT, this is One Hot Encoded, Embedded, Positional Encoded and so on. Anyway we have a matrix, still after attention mechanism we have a matrix, how we end with a probability vector and not with a probability matrix?
  
  **Tags:** chatgpt
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-06-04. Score: 1
  
  Once the last level of the [*multi-head attention*](https://paperswithcode.com/method/multi-head-attention) mechanism returns its output, the matrix is flattened in an array and sent as input to a *[linear](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear)* level (a.k.a. a *fully-connected* layer).The output of this layer is activated via [Softmax](https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html#softmax) in order to have probabilities associated to the next token. The token with the highest probability is then returned.
  
  
  ---
  
  