<p>No, it's actually a very precise guess determined by relatively complex statistical equations.</p>
<p>GPT-3 is able to take commands because it has been trained on a dataset of commands. This means that it is able to recognize patterns in sequences of tokens that indicate a command. In the example input, the most likely prediction is to insert a period and end the sentence because this is what typically happens at the end of a sentence. However, GPT-3 is able to take commands because it is able to recognize patterns that indicate a different kind of sequence, such as a command.</p>

CC BY-SA 4.0
2022-10-17T10:48:03.933Z
2022-10-17T10:48:03.933Z
Faizy
<p>The dataset has been curated in such a way as that it believes the next most likely set of tokens are answers.</p>
<p>i.e. Its been shown that after a command surrounded by special tokens the next tokens are always responses to the commands.</p>

CC BY-SA 4.0
2024-11-07T15:49:33.263Z
2024-11-07T15:49:33.263Z
Leonhard Piff
<p>From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the statistically most likely prediction be to insert a period and end the sentence?</p>
<pre><code>Input: write me a beautiful sentence

Output: I cannot put into words how much I love you, so I'll just say it's infinite.
</code></pre>

<natural-language-processing><transformer><gpt-3>
If GPT-3 is trained on predicting the next token, how is it able to take commands?
Andrew Tang
# If GPT-3 is trained on predicting the next token, how is it able to take commands?

(Asked by: username_0 on 2022-10-04)

From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the statistically most likely prediction be to insert a period and end the sentence?
Input: write me a beautiful sentence

Output: I cannot put into words how much I love you, so I'll just say it's infinite.

**Tags:** natural-language-processing, transformer, gpt-3


---

## 2 answers

### Answer by username_1 on 2022-10-17. Score: 1

No, it's actually a very precise guess determined by relatively complex statistical equations.

GPT-3 is able to take commands because it has been trained on a dataset of commands. This means that it is able to recognize patterns in sequences of tokens that indicate a command. In the example input, the most likely prediction is to insert a period and end the sentence because this is what typically happens at the end of a sentence. However, GPT-3 is able to take commands because it is able to recognize patterns that indicate a different kind of sequence, such as a command.


---

### Answer by username_2 on 2024-11-07. Score: 1

The dataset has been curated in such a way as that it believes the next most likely set of tokens are answers.

i.e. Its been shown that after a command surrounded by special tokens the next tokens are always responses to the commands.


---

