Answers.Body: <p>No, it's actually a very precise guess determined by relatively complex statistical equations.</p>
  <p>GPT-3 is able to take commands because it has been trained on a dataset of commands. This means that it is able to recognize patterns in sequences of tokens that indicate a command. In the example input, the most likely prediction is to insert a period and end the sentence because this is what typically happens at the end of a sentence. However, GPT-3 is able to take commands because it is able to recognize patterns that indicate a different kind of sequence, such as a command.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T10:48:03.933Z
Answers.LastActivityDate: 2022-10-17T10:48:03.933Z
Answers.UserDisplayName: Faizy
Answers.Body: <p>The dataset has been curated in such a way as that it believes the next most likely set of tokens are answers.</p>
  <p>i.e. Its been shown that after a command surrounded by special tokens the next tokens are always responses to the commands.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-07T15:49:33.263Z
Answers.LastActivityDate: 2024-11-07T15:49:33.263Z
Answers.UserDisplayName: Leonhard Piff
Body: <p>From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the statistically most likely prediction be to insert a period and end the sentence?</p>
  <pre><code>Input: write me a beautiful sentence
  
  Output: I cannot put into words how much I love you, so I'll just say it's infinite.
  </code></pre>
  
Tags: <natural-language-processing><transformer><gpt-3>
Title: If GPT-3 is trained on predicting the next token, how is it able to take commands?
UserDisplayName: Andrew Tang
ThreadText: # If GPT-3 is trained on predicting the next token, how is it able to take commands?
  
  (Asked by: username_0 on 2022-10-04)
  
  From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the statistically most likely prediction be to insert a period and end the sentence?
  
  ```
  Input: write me a beautiful sentence
  
  Output: I cannot put into words how much I love you, so I'll just say it's infinite.
  
  ```
  
  **Tags:** natural-language-processing, transformer, gpt-3
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-10-17. Score: 1
  
  No, it's actually a very precise guess determined by relatively complex statistical equations.
  
  GPT-3 is able to take commands because it has been trained on a dataset of commands. This means that it is able to recognize patterns in sequences of tokens that indicate a command. In the example input, the most likely prediction is to insert a period and end the sentence because this is what typically happens at the end of a sentence. However, GPT-3 is able to take commands because it is able to recognize patterns that indicate a different kind of sequence, such as a command.
  
  
  ---
  
  ### Answer by username_2 on 2024-11-07. Score: 1
  
  The dataset has been curated in such a way as that it believes the next most likely set of tokens are answers.
  
  i.e. Its been shown that after a command surrounded by special tokens the next tokens are always responses to the commands.
  
  
  ---
  
  