Answers.Body: <p>Yes, this is actually a limitation known as <a href="https://en.wikipedia.org/wiki/Catastrophic_interference" rel="nofollow noreferrer">catastrophic forgetting</a>. 
  A proposed way to deal with this is elastic weight consolidation that "remembers old tasks by selectively slowing down learning on the weights important for those tasks". See <a href="https://arxiv.org/abs/1612.00796" rel="nofollow noreferrer">Overcoming catastrophic forgetting in neural networks</a> for details. Another approach is <a href="https://arxiv.org/pdf/1606.09282.pdf" rel="nofollow noreferrer">Learning without forgetting</a>.</p>
  
  <p>If the tasks are different, the approach you are talking about is called transfer learning. You might want to have a look at multi-task learning as well </p>
  
  <p>If the tasks are the same, you could try creating a join of both datasets and training on that.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-27T12:19:18.750Z
Answers.LastActivityDate: 2019-11-28T14:02:16.067Z
Answers.LastEditDate: 2019-11-28T14:02:16.067Z
Answers.UserDisplayName: cookiemonster
Body: <p>I read <a href="https://hmkcode.com/ai/backpropagation-step-by-step/" rel="nofollow noreferrer">this tutorial</a> about backpropagation.</p>
  
  <p>So using this backpropagation we are training the neural network repeatedly for one input set, say [2,4], until we reach 100% accuracy of getting 1 as output. And the neural network is adjusting its weight values accordingly. So once after the neural network is trained this way, suppose we are giving another input set, say [6,8], also then will the neural network update its weight values (overwriting previous values), right? This will result in losing the previous learning, right?</p>
  
Tags: <neural-networks><backpropagation><artificial-neuron><sigmoid>
Title: How can I train a neural network for another input set, without losing the learning of the previous input set?
UserDisplayName: Tyrostir
ThreadText: # How can I train a neural network for another input set, without losing the learning of the previous input set?
  
  (Asked by: username_0 on 2019-11-27)
  
  I read [this tutorial](https://hmkcode.com/ai/backpropagation-step-by-step/) about backpropagation.
  
  So using this backpropagation we are training the neural network repeatedly for one input set, say [2,4], until we reach 100% accuracy of getting 1 as output. And the neural network is adjusting its weight values accordingly. So once after the neural network is trained this way, suppose we are giving another input set, say [6,8], also then will the neural network update its weight values (overwriting previous values), right? This will result in losing the previous learning, right?
  
  **Tags:** neural-networks, backpropagation, artificial-neuron, sigmoid
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-27. Score: 3
  
  Yes, this is actually a limitation known as [catastrophic forgetting](https://en.wikipedia.org/wiki/Catastrophic_interference).
  A proposed way to deal with this is elastic weight consolidation that "remembers old tasks by selectively slowing down learning on the weights important for those tasks". See [Overcoming catastrophic forgetting in neural networks](https://arxiv.org/abs/1612.00796) for details. Another approach is [Learning without forgetting](https://arxiv.org/pdf/1606.09282.pdf).
  
  If the tasks are different, the approach you are talking about is called transfer learning. You might want to have a look at multi-task learning as well
  
  If the tasks are the same, you could try creating a join of both datasets and training on that.
  
  
  ---
  
  