Answers.Body: <p>You are looking for <a href="https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf" rel="nofollow noreferrer">incremental (or online) learning</a>.</p>
  
  <p>A CNN can be trained incrementally. For example, in the paper <a href="https://pdfs.semanticscholar.org/8f21/c99d8257c79baf22c211ed17a2224574b524.pdf" rel="nofollow noreferrer">Incremental Learning of Convolutional Neural Networks</a>, the authors propose an incremental learning algorithm (inspired by <a href="https://en.wikipedia.org/wiki/AdaBoost" rel="nofollow noreferrer">AdaBoost</a> and <a href="https://www.researchgate.net/publication/2489080_Learn_An_Incremental_Learning_Algorithm_for_Supervised_Neural_Networks" rel="nofollow noreferrer">Learn++</a>, which is another incremental learning algorithm for supervised learning of neural networks) for CNNs. </p>
  
  <p>However, note that incremental learning is a challenging task, given the  <em>stability-plasticity dilemma</em>: a completely <em>stable</em> model, in order to keep being stable, will attempt to preserve the existing knowledge, so it will not learn new knowledge; similarly, a completely <em>plastic</em> model, in order to keep being plastic, it will keep forgetting previously acquired knowledge so that to learn new information.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-31T16:21:04.090Z
Answers.LastActivityDate: 2019-05-31T16:21:04.090Z
Answers.UserDisplayName: nbro
Body: <p>Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2,  Instead of cleaning all of learned memory, and relearn handwriting data 0, 1, and 2 at the same time. </p>
  
  <p>Can CNN do the same thing? Can CNN learn something new, but keep the previous memory? If yes, the efficiency could be high.  Right now, I have to give all of data at the same time, the efficiency is very very low.</p>
  
Tags: <machine-learning><convolutional-neural-networks><incremental-learning><online-learning>
Title: Can a CNN be trained incrementally?
UserDisplayName: Sunson29
ThreadText: # Can a CNN be trained incrementally?
  
  (Asked by: username_0 on 2019-05-31)
  
  Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all of learned memory, and relearn handwriting data 0, 1, and 2 at the same time.
  
  Can CNN do the same thing? Can CNN learn something new, but keep the previous memory? If yes, the efficiency could be high. Right now, I have to give all of data at the same time, the efficiency is very very low.
  
  **Tags:** machine-learning, convolutional-neural-networks, incremental-learning, online-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-31. Score: 4
  
  You are looking for [incremental (or online) learning](https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf).
  
  A CNN can be trained incrementally. For example, in the paper [Incremental Learning of Convolutional Neural Networks](https://pdfs.semanticscholar.org/8f21/c99d8257c79baf22c211ed17a2224574b524.pdf), the authors propose an incremental learning algorithm (inspired by [AdaBoost](https://en.wikipedia.org/wiki/AdaBoost) and [Learn++](https://www.researchgate.net/publication/2489080_Learn_An_Incremental_Learning_Algorithm_for_Supervised_Neural_Networks), which is another incremental learning algorithm for supervised learning of neural networks) for CNNs.
  
  However, note that incremental learning is a challenging task, given the *stability-plasticity dilemma*: a completely *stable* model, in order to keep being stable, will attempt to preserve the existing knowledge, so it will not learn new knowledge; similarly, a completely *plastic* model, in order to keep being plastic, it will keep forgetting previously acquired knowledge so that to learn new information.
  
  
  ---
  
  