Answers.Body: <p>Per Neil's line of questioning:</p>
  
  <ul>
  <li><p>If you're doing offline, supervised or reinforcement learning, where you have the dataset and decide what to show to the network next, there's no reason to do this. It's better to just adjust your training to show rare examples to the network more often.</p></li>
  <li><p>If you're doing online reinforcement learning, then the agent typically controls which examples it sees next through its choice of actions. Increasing the agent's exploration rate will cause it to see a more diverse set of examples, but these examples are also less likely to be useful for solving the task the agent is working on.</p></li>
  <li><p>If you're doing supervised, online, learning, and you don't control the order in which examples appear, it might be useful to freeze some of the weights, but it might be better, and would certainly be simpler, to increase the learning rate for rare classes instead (in effect, showing the network a given rare example repeatedly when it <em>does</em> show up, so that it takes longer to forget it).</p></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-14T14:22:00.670Z
Answers.LastActivityDate: 2018-10-14T14:22:00.670Z
Answers.UserDisplayName: John Doucette
Body: <p>Would this work at all?</p>
  
  <p>Idea is to start training a neural net with some number of nodes. Then, add some new nodes and more layers and start training only the new nodes (or only modifying the old nodes very slightly). Ideally, we would connect all old nodes to the new layer added since we might have learned many useful things in the hidden layers. Then repeat this many times.</p>
  
  <p>Intuition is that if the old nodes give bad information the new layer of nodes will weight the activations of old nodes close to zero and learn new/better concepts in the new nodes. The benefit is that we will keep old knowledge forever.</p>
  
  <p>Caveat is that the network can still temporarily "forget" concepts if a new layer weights old information close to zero, but it can potentially remember it again too.</p>
  
  <p>If this completely fails, I'm curious if there's some known way to prevent a neural network from forgetting concepts it learned.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-11T07:08:00.070Z
Comments.Text: Could you clarify what kind of forgetting scenario you hope this will address? The problem usually occurs in online learning due to temporal sampling bias or population drift. At what point would you add new nodes/layers?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-12T06:42:39.000Z
Comments.Text: The scenario is that when training a network, you might end up with catastrophic forgetting. That is, if you haven't seen an input in a while, the changes in node weights might make it forget how to recognize the input.
Comments.UserDisplayName: user3721127
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-12T06:45:34.810Z
Comments.Text: There's a big difference between addressing this issue in offline and online training, or in Reinforcement Learning. I most often see the term "catastrophic forgetting" in RL, is that what you are asking about?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-14T03:32:34.797Z
Comments.Text: I'm referring to forgetting in the following sense: If you train a neural network to recognize digits from 0-9, and if you show it no examples of the number 5 for a very long time (but all the other digits), it will forget how to recognize it as the neural network weights change.
Comments.UserDisplayName: user3721127
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-14T07:19:08.850Z
Comments.Text: This is getting a little circular: I am asking about the higher-level scenario, not the data flow. I.e. "How does it *occur* that you have a training scenario where relevant data is not available?". For an offline supervised learning on MNIST using the data set, this is not going to happen, and is also trivial to fix if it is made to occur artificially, as there are no real consequences to a dip in performance. To analyse the consequences of your idea requires understanding the context in which it will be used.
Comments.UserDisplayName: Neil Slater
Tags: <neural-networks>
Title: Would this work to prevent forgetting: train a neural net with N nodes. Then, add more nodes and stop training the original nodes
UserDisplayName: user3721127
ThreadText: # Would this work to prevent forgetting: train a neural net with N nodes. Then, add more nodes and stop training the original nodes
  
  (Asked by: username_0 on 2018-10-11)
  
  Would this work at all?
  
  Idea is to start training a neural net with some number of nodes. Then, add some new nodes and more layers and start training only the new nodes (or only modifying the old nodes very slightly). Ideally, we would connect all old nodes to the new layer added since we might have learned many useful things in the hidden layers. Then repeat this many times.
  
  Intuition is that if the old nodes give bad information the new layer of nodes will weight the activations of old nodes close to zero and learn new/better concepts in the new nodes. The benefit is that we will keep old knowledge forever.
  
  Caveat is that the network can still temporarily "forget" concepts if a new layer weights old information close to zero, but it can potentially remember it again too.
  
  If this completely fails, I'm curious if there's some known way to prevent a neural network from forgetting concepts it learned.
  
  **Tags:** neural-networks
  > **Comments:**
  > **username_1**: Could you clarify what kind of forgetting scenario you hope this will address? The problem usually occurs in online learning due to temporal sampling bias or population drift. At what point would you add new nodes/layers?  
  > **username_0**: The scenario is that when training a network, you might end up with catastrophic forgetting. That is, if you haven't seen an input in a while, the changes in node weights might make it forget how to recognize the input.  
  > **username_1**: There's a big difference between addressing this issue in offline and online training, or in Reinforcement Learning. I most often see the term "catastrophic forgetting" in RL, is that what you are asking about?  
  > **username_0**: I'm referring to forgetting in the following sense: If you train a neural network to recognize digits from 0-9, and if you show it no examples of the number 5 for a very long time (but all the other digits), it will forget how to recognize it as the neural network weights change.  
  > **username_1**: This is getting a little circular: I am asking about the higher-level scenario, not the data flow. I.e. "How does it *occur* that you have a training scenario where relevant data is not available?". For an offline supervised learning on MNIST using the data set, this is not going to happen, and is also trivial to fix if it is made to occur artificially, as there are no real consequences to a dip in performance. To analyse the consequences of your idea requires understanding the context in which it will be used.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-10-14. Score: 2 | Accepted answer
  
  Per Neil's line of questioning:
  
  * If you're doing offline, supervised or reinforcement learning, where you have the dataset and decide what to show to the network next, there's no reason to do this. It's better to just adjust your training to show rare examples to the network more often.
  * If you're doing online reinforcement learning, then the agent typically controls which examples it sees next through its choice of actions. Increasing the agent's exploration rate will cause it to see a more diverse set of examples, but these examples are also less likely to be useful for solving the task the agent is working on.
  * If you're doing supervised, online, learning, and you don't control the order in which examples appear, it might be useful to freeze some of the weights, but it might be better, and would certainly be simpler, to increase the learning rate for rare classes instead (in effect, showing the network a given rare example repeatedly when it *does* show up, so that it takes longer to forget it).
  
  
  ---
  
  