Answers.Body: <p>In theory, there is nothing stopping you from updating the weights of a neural network whenever you like. You run an example through the network, calculate the difference between the network's output and the answer you expected, and run back propagation, exactly the same as you do when you initially train the network. Of course, usually networks are trained with large batches of data instead of single examples at a time, so if you wanted to do a weight update you should save up a bunch of data and pass through a batch (ideally the same batch size that you used during training, though there's nothing stopping you from passing in different batch sizes).</p>
  <p>Keep in mind this is all theoretical. In practice, adjusting the weights on a deployed network will probably be very difficult because the models weights have been exported in a format optimized for inference. And it's better to have distinct releases with sets of weights that do not change rather than continuously updating the same model.</p>
  <p>Either way, changing the weights continuously would not affect the &quot;memory&quot; of the network in any way.  The lengths of sequences that sequence-to-sequence models like transformers or RNNs can accept is an entirely separate parameter.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-09T00:56:02.563Z
Answers.Text: Thanks for all the info. The only thing I don't get is why changing the weights continuously wouldn't allow it to have memory and learn new things. Isn't that how humans learn; by changing the weights of some specific neurons?
Answers.UserDisplayName: MaiaVictor
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-09T06:37:48.507Z
Answers.Text: GTP-3 is trained on 45 TB data, so any additional examples you might give it will not make much difference https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/
Answers.UserDisplayName: Stefan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-09T13:16:51.083Z
Answers.Text: @MaiaVictor The memory in sequence-to-sequence models is more akin to working memory in humans: memory that is stored not in neuron synapses, but in the activity of neuron. The long term synaptic memory you're referring to is used by the network to learn the overall task. But this doesn't mean changing the weights will let you feed in arbitrary length sequences: those sequences are stored in the activity of the network's neurons. In RNNs, this working memory is modeled explicitly by gates in each neuron (look up LSTM or GRU). In transformers, it's much more complicated.
Answers.UserDisplayName: xojfqa
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-09T13:18:56.100Z
Answers.Text: Keep in mind that while what Stefan said is true, you *can* make a big difference in the quality of GPT3's: not by training the network itself on new examples, but by including examples at the start of your prompt. This is called prompt engineering, good examples are available here https://beta.openai.com/examples/
Answers.UserDisplayName: xojfqa
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-26T20:19:33.050Z
Answers.Text: @Stefan isn't this just strongly hinting that training is terribly inefficient? Humans have seen way more than 45 TB of raw data in their lifetime, yet they can read a single sentence such as "jellyfish are biologically immortal" and learn a new concept for the rest of their life. Somehow humans are extremely good at storing new "insightful" information they're exposed very briefly, while ignoring massive "bland" amount of information (like all you see while commuting).
Answers.UserDisplayName: MaiaVictor
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-08T19:05:32.927Z
Answers.LastActivityDate: 2022-10-08T19:10:37.427Z
Answers.LastEditDate: 2022-10-08T19:10:37.427Z
Answers.UserDisplayName: xojfqa
Answers.Body: <p>With <span class="math-container">$175$</span> Billion parameters, <code>GPT-3</code> is remarkably large and powerful, but it has several limitations and risks associated with its usage. <a href="https://www.techtarget.com/searchenterpriseai/definition/GPT-3#:%7E:text=While%20GPT-3%20is%20remarkably%20large%20and%20powerful%2C%20it,ongoing%20long-term%20memory%20that%20learns%20from%20each%20interaction." rel="nofollow noreferrer"><strong>The biggest issue is that GPT-3 <code>can't</code> continuously learn once trained?</strong></a>. It has been <strong>pre-trained</strong>(as the name ~ Generative Pre-trained Transformer), which means that <strong>it doesn't have an ongoing long-term memory that learns from each interaction</strong>.</p>
  <p>In addition, <code>GPT-3</code> suffers from the same problems as all <code>Neural Networks</code>: their lack of ability to explain and interpret <em>why certain inputs result in specific outputs</em>.</p>
  <p>Another reason could be that the model has reached a point of diminishing returns, meaning that any additional training is unlikely to result in significant improvements.</p>
  <blockquote>
  <p>&quot;A significant concern when building AI models like these is <strong><a href="https://medium.com/analytics-vidhya/a-simple-explanation-of-gpt-3-571aca61208c" rel="nofollow noreferrer">diminishing returns</a></strong>—that is, you cannot simply scale the model up forever. At some point, some factor(s) of the model will <strong>plateau</strong>, whether it’s the <strong>information generated</strong>, the <strong>dataset size</strong>, the <strong>training regime</strong>, etc&quot;.</p>
  </blockquote>
  <p>However, at the level of <code>GPT-2</code>, there was no indication that this plateau had been reached. Thus, the “bigger and better” tactic continued, bringing us <code>GPT-3</code>&quot;. So, it may also be possible that the model has simply reached a plateau in its learning and is unable to make any further progress.</p>
  <hr />
  <p><strong>References:</strong></p>
  <ul>
  <li><a href="https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/" rel="nofollow noreferrer">OpenAI GPT-3: Everything You Need to Know</a></li>
  <li><a href="https://medium.com/analytics-vidhya/a-simple-explanation-of-gpt-3-571aca61208c" rel="nofollow noreferrer">an overview of GPT-3: AI of the future</a></li>
  <li><a href="https://www.techtarget.com/searchenterpriseai/definition/GPT-3#:%7E:text=While%20GPT-3%20is%20remarkably%20large%20and%20powerful%2C%20it,ongoing%20long-term%20memory%20that%20learns%20from%20each%20interaction." rel="nofollow noreferrer">GPT-3</a></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-21T21:51:54.703Z
Answers.LastActivityDate: 2022-10-21T22:34:00.183Z
Answers.LastEditDate: 2022-10-21T22:34:00.183Z
Answers.UserDisplayName: Faizy
Body: <p>GPT-3 has a prompt limit of about ~2048 &quot;tokens&quot;, which corresponds to about 4 characters in text. If my understanding is correct, a deep neural network is not learning after it is trained and is used to produce an output, and, as such, this limitation comes from amount of the input neurons. My question is: <strong>what is stopping us from using the same algorithm we use for training, when using the network?</strong> That would allow it to adjust its weights and, in a way, provide a form of long-term memory which could let it handle prompts with arbitrarily long limits. Is my line of thinking worng?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-08T15:10:02.963Z
Comments.Text: I'm not well acquainted with machine learning and the involved literature, so this is a very newbie question. I tried to make it as high quality as I could, but I'm certainly limited on how well I can make questions about a subject I don't know. Hope this kind if question is acceptable here.
Comments.UserDisplayName: MaiaVictor
Tags: <deep-learning><language-model>
Title: Why can't language models, like GPT-3, continuously learn once trained?
UserDisplayName: MaiaVictor
ThreadText: # Why can't language models, like GPT-3, continuously learn once trained?
  
  (Asked by: username_0 on 2022-10-08)
  
  GPT-3 has a prompt limit of about ~2048 "tokens", which corresponds to about 4 characters in text. If my understanding is correct, a deep neural network is not learning after it is trained and is used to produce an output, and, as such, this limitation comes from amount of the input neurons. My question is: **what is stopping us from using the same algorithm we use for training, when using the network?** That would allow it to adjust its weights and, in a way, provide a form of long-term memory which could let it handle prompts with arbitrarily long limits. Is my line of thinking worng?
  
  **Tags:** deep-learning, language-model
  
  > **Comments:**
  > **username_0**: I'm not well acquainted with machine learning and the involved literature, so this is a very newbie question. I tried to make it as high quality as I could, but I'm certainly limited on how well I can make questions about a subject I don't know. Hope this kind if question is acceptable here.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-10-08. Score: 5
  
  In theory, there is nothing stopping you from updating the weights of a neural network whenever you like. You run an example through the network, calculate the difference between the network's output and the answer you expected, and run back propagation, exactly the same as you do when you initially train the network. Of course, usually networks are trained with large batches of data instead of single examples at a time, so if you wanted to do a weight update you should save up a bunch of data and pass through a batch (ideally the same batch size that you used during training, though there's nothing stopping you from passing in different batch sizes).
  
  Keep in mind this is all theoretical. In practice, adjusting the weights on a deployed network will probably be very difficult because the models weights have been exported in a format optimized for inference. And it's better to have distinct releases with sets of weights that do not change rather than continuously updating the same model.
  
  Either way, changing the weights continuously would not affect the "memory" of the network in any way. The lengths of sequences that sequence-to-sequence models like transformers or RNNs can accept is an entirely separate parameter.
  
  > **Comments:**
  > **username_0**: Thanks for all the info. The only thing I don't get is why changing the weights continuously wouldn't allow it to have memory and learn new things. Isn't that how humans learn; by changing the weights of some specific neurons?  
  > **username_2**: GTP-3 is trained on 45 TB data, so any additional examples you might give it will not make much difference https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/  
  > **username_1**: @username_0 The memory in sequence-to-sequence models is more akin to working memory in humans: memory that is stored not in neuron synapses, but in the activity of neuron. The long term synaptic memory you're referring to is used by the network to learn the overall task. But this doesn't mean changing the weights will let you feed in arbitrary length sequences: those sequences are stored in the activity of the network's neurons. In RNNs, this working memory is modeled explicitly by gates in each neuron (look up LSTM or GRU). In transformers, it's much more complicated.  
  > **username_1**: Keep in mind that while what username_2 said is true, you *can* make a big difference in the quality of GPT3's: not by training the network itself on new examples, but by including examples at the start of your prompt. This is called prompt engineering, good examples are available here https://beta.openai.com/examples/  
  > **username_0**: @username_2 isn't this just strongly hinting that training is terribly inefficient? Humans have seen way more than 45 TB of raw data in their lifetime, yet they can read a single sentence such as "jellyfish are biologically immortal" and learn a new concept for the rest of their life. Somehow humans are extremely good at storing new "insightful" information they're exposed very briefly, while ignoring massive "bland" amount of information (like all you see while commuting).  
  
  ---
  
  ### Answer by username_3 on 2022-10-21. Score: 1
  
  With $175$ Billion parameters, `GPT-3` is remarkably large and powerful, but it has several limitations and risks associated with its usage. [**The biggest issue is that GPT-3 `can't` continuously learn once trained?**](https://www.techtarget.com/searchenterpriseai/definition/GPT-3#:%7E:text=While%20GPT-3%20is%20remarkably%20large%20and%20powerful%2C%20it,ongoing%20long-term%20memory%20that%20learns%20from%20each%20interaction.). It has been **pre-trained**(as the name ~ Generative Pre-trained Transformer), which means that **it doesn't have an ongoing long-term memory that learns from each interaction**.
  
  In addition, `GPT-3` suffers from the same problems as all `Neural Networks`: their lack of ability to explain and interpret *why certain inputs result in specific outputs*.
  
  Another reason could be that the model has reached a point of diminishing returns, meaning that any additional training is unlikely to result in significant improvements.
  
  > "A significant concern when building AI models like these is **[diminishing returns](https://medium.com/analytics-vidhya/a-simple-explanation-of-gpt-3-571aca61208c)**—that is, you cannot simply scale the model up forever. At some point, some factor(s) of the model will **plateau**, whether it’s the **information generated**, the **dataset size**, the **training regime**, etc".
  
  However, at the level of `GPT-2`, there was no indication that this plateau had been reached. Thus, the “bigger and better” tactic continued, bringing us `GPT-3`". So, it may also be possible that the model has simply reached a plateau in its learning and is unable to make any further progress.
  
  ---
  
  **References:**
  
  * [OpenAI GPT-3: Everything You Need to Know](https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/)
  * [an overview of GPT-3: AI of the future](https://medium.com/analytics-vidhya/a-simple-explanation-of-gpt-3-571aca61208c)
  * [GPT-3](https://www.techtarget.com/searchenterpriseai/definition/GPT-3#:%7E:text=While%20GPT-3%20is%20remarkably%20large%20and%20powerful%2C%20it,ongoing%20long-term%20memory%20that%20learns%20from%20each%20interaction.)
  
  
  ---
  
  