Answers.Body: <p>There are several terms or expressions related to such systems, such as</p>
  <ul>
  <li><a href="https://web.archive.org/web/20200709121648/https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf" rel="nofollow noreferrer">online learning</a></li>
  <li><a href="https://web.archive.org/web/20200709121648/https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf" rel="nofollow noreferrer">incremental learning</a></li>
  <li>continuous learning</li>
  <li><a href="https://wiki.continualai.org/" rel="nofollow noreferrer">continual learning</a></li>
  <li><a href="https://wiki.continualai.org/" rel="nofollow noreferrer">lifelong learning</a></li>
  </ul>
  <p>They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.</p>
  <p>The opposite of online is offline. However, the expression <a href="https://stats.stackexchange.com/q/70761/82135"><em>batch learning</em></a> is sometimes used as an antonym for online learning.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-25T11:14:24.483Z
Answers.LastActivityDate: 2024-01-02T13:42:03.987Z
Answers.LastEditDate: 2024-01-02T13:42:03.987Z
Answers.UserDisplayName: nbro
Answers.Body: <p>In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies.</p>
  
  <p>Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the same type of <strong>learning during operation</strong>. And this gives as cues to dig deeper into the topic.</p>
  
  <p>The easiest way to learn during operation is simple reinforcement learning. DQN of Deepmind works exactly that way. It tries out different moves and learns what works by receiving feedback through the reward function. There is not necessarily a separate learning and working phase. The AI just gets better over time. This means, that the NN keeps adapting to the challenge at hand and if that challenge changes, it adopts to the new challenge over time and "forgets" what worked before, if this old strategy is no longer efficient.</p>
  
  <p>Your first reference to Deepmind addresses the aspect of forgetting. When talking about continuous learning, it could also mean, that a system does <strong>not</strong> forget previous skills but can utilize them later after learning something totally different. This requires more advanced architectures of AI. And this approach doesn't mean, that the NN must necessarily keep learning during operation. It could use this technique during training and stop learning once it is productive.</p>
  
  <p>And there is a third field, recursive networks, which do not pass their information from neuron to neuron in a straight fashion but can contain loops or other types of gates, that control the flow of information. Such architectures are usually more complex than normal NNs, but resemble the working of animal brains a little more. In this field the term <strong>dynamic</strong> is used frequently.</p>
  
  <p>Therefore, I would say that the term you are looking for is <strong>continuous learning</strong> - I have also read <strong>online learning</strong> for this type of NN - when you want to say that the NN keeps learning during production. The term <strong>dynamic</strong> is more common for recursive or other more complex types of NNs and fits there better in my opinion.</p>
  
  <p>To conclude my answer, I need to jump back to the beginning. Many terms in the field of AI are not finally settled. Therefore my analysis is just a summary of the trends I have seen in different publications and lectures and must not hold true in the years to come. Hope it helps nevertheless.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-29T20:30:37.060Z
Answers.LastActivityDate: 2017-08-29T20:30:37.060Z
Answers.UserDisplayName: Demento
Answers.Body: <p>ML, being a relatively young and fast-developing field, has numerous (near-)synonyms for many concepts.</p>
  <p>One paradigm difference is whether a model is learned from a static, pre-defined set of data, or whether it adapts as new data is presented to it over time.</p>
  <p>Some of the terms used to describe these two paradigms respectively (with subtle differences in meaning between authors/terms) are:</p>
  <ul>
  <li>Offline / batch / <a href="https://www.cs.uic.edu/%7Eliub/lifelong-learning/introduction-lifelong-learning.pdf" rel="nofollow noreferrer">isolated</a> learning</li>
  <li>Online / continual / continuous / incremental / lifelong learning</li>
  </ul>
  <p>Further, some familiar branches of ML (like Transfer Learning and Multi-task learning) have a lot of intersection with Continual Learning.</p>
  <hr />
  <sup>
  <p>Related q's:</p>
  <ul>
  <li><p><a href="https://ai.stackexchange.com/questions/23226/what-is-the-difference-between-active-learning-and-online-learning">What is the difference between active learning and online learning?</a></p>
  </li>
  <li><p><a href="https://stats.stackexchange.com/questions/897/online-vs-offline-learning">https://stats.stackexchange.com/questions/897/online-vs-offline-learning</a>
  </sup></p>
  </li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-30T11:09:40.210Z
Answers.LastActivityDate: 2020-11-09T11:43:24.067Z
Answers.LastEditDate: 2020-11-09T11:43:24.067Z
Answers.UserDisplayName: brazofuerte
Body: <p>As I understand it from this <a href="http://videolectures.net/deeplearning2015_vincent_machine_learning/" rel="nofollow noreferrer">video lecture</a>, there are three types of deep learning:</p>
  
  <ul>
  <li>Supervised</li>
  <li>Unsupervised</li>
  <li>Reinforcement</li>
  </ul>
  
  <p>All these can serve to train a neural network either <em>only prior</em> to its deployment or <em>during</em> its operating. </p>
  
  <p>For the latter case, it is referred to as <em>continuous</em> learning <a href="http://deepmind.com/blog/enabling-continual-learning-in-neural-networks/" rel="nofollow noreferrer">here</a> and <a href="http://medium.com/@Synced/deep-learning-in-real-time-inference-acceleration-and-continuous-training-17dac9438b0b" rel="nofollow noreferrer">here</a> and as <em>dynamic</em> learning <a href="http://conferences.oreilly.com/artificial-intelligence/ai-ny/public/schedule/detail/59511" rel="nofollow noreferrer">here</a> and <a href="http://www.datasciencecentral.com/profiles/blogs/static-dynamical-machine-learning-what-is-the-difference" rel="nofollow noreferrer">here</a>.</p>
  
  <p>Which term should I use to refer to a machine learning algorithm that keeps on learning (even after deployment)? If it's "continuous", is there an opposing term (such as "static" for "dynamic") for those systems that stop learning before being deployed? </p>
  
Tags: <neural-networks><deep-learning><terminology><incremental-learning>
Title: What do you call a machine learning system that keeps on learning?
UserDisplayName: ZakC
ThreadText: # What do you call a machine learning system that keeps on learning?
  
  (Asked by: username_0 on 2017-08-29)
  
  As I understand it from this [video lecture](http://videolectures.net/deeplearning2015_vincent_machine_learning/), there are three types of deep learning:
  
  * Supervised
  * Unsupervised
  * Reinforcement
  
  All these can serve to train a neural network either *only prior* to its deployment or *during* its operating.
  
  For the latter case, it is referred to as *continuous* learning [here](http://deepmind.com/blog/enabling-continual-learning-in-neural-networks/) and [here](http://medium.com/@Synced/deep-learning-in-real-time-inference-acceleration-and-continuous-training-17dac9438b0b) and as *dynamic* learning [here](http://conferences.oreilly.com/artificial-intelligence/ai-ny/public/schedule/detail/59511) and [here](http://www.datasciencecentral.com/profiles/blogs/static-dynamical-machine-learning-what-is-the-difference).
  
  Which term should I use to refer to a machine learning algorithm that keeps on learning (even after deployment)? If it's "continuous", is there an opposing term (such as "static" for "dynamic") for those systems that stop learning before being deployed?
  
  **Tags:** neural-networks, deep-learning, terminology, incremental-learning
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2019-06-25. Score: 7
  
  There are several terms or expressions related to such systems, such as
  
  * [online learning](https://web.archive.org/web/20200709121648/https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf)
  * [incremental learning](https://web.archive.org/web/20200709121648/https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-19.pdf)
  * continuous learning
  * [continual learning](https://wiki.continualai.org/)
  * [lifelong learning](https://wiki.continualai.org/)
  
  They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.
  
  The opposite of online is offline. However, the expression [*batch learning*](https://stats.stackexchange.com/q/70761/82135) is sometimes used as an antonym for online learning.
  
  
  ---
  
  ### Answer by username_2 on 2017-08-29. Score: 1
  
  In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies.
  
  Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the same type of **learning during operation**. And this gives as cues to dig deeper into the topic.
  
  The easiest way to learn during operation is simple reinforcement learning. DQN of Deepmind works exactly that way. It tries out different moves and learns what works by receiving feedback through the reward function. There is not necessarily a separate learning and working phase. The AI just gets better over time. This means, that the NN keeps adapting to the challenge at hand and if that challenge changes, it adopts to the new challenge over time and "forgets" what worked before, if this old strategy is no longer efficient.
  
  Your first reference to Deepmind addresses the aspect of forgetting. When talking about continuous learning, it could also mean, that a system does **not** forget previous skills but can utilize them later after learning something totally different. This requires more advanced architectures of AI. And this approach doesn't mean, that the NN must necessarily keep learning during operation. It could use this technique during training and stop learning once it is productive.
  
  And there is a third field, recursive networks, which do not pass their information from neuron to neuron in a straight fashion but can contain loops or other types of gates, that control the flow of information. Such architectures are usually more complex than normal NNs, but resemble the working of animal brains a little more. In this field the term **dynamic** is used frequently.
  
  Therefore, I would say that the term you are looking for is **continuous learning** - I have also read **online learning** for this type of NN - when you want to say that the NN keeps learning during production. The term **dynamic** is more common for recursive or other more complex types of NNs and fits there better in my opinion.
  
  To conclude my answer, I need to jump back to the beginning. Many terms in the field of AI are not finally settled. Therefore my analysis is just a summary of the trends I have seen in different publications and lectures and must not hold true in the years to come. Hope it helps nevertheless.
  
  
  ---
  
  ### Answer by username_3 on 2020-10-30. Score: 1
  
  ML, being a relatively young and fast-developing field, has numerous (near-)synonyms for many concepts.
  
  One paradigm difference is whether a model is learned from a static, pre-defined set of data, or whether it adapts as new data is presented to it over time.
  
  Some of the terms used to describe these two paradigms respectively (with subtle differences in meaning between authors/terms) are:
  
  * Offline / batch / [isolated](https://www.cs.uic.edu/%7Eliub/lifelong-learning/introduction-lifelong-learning.pdf) learning
  * Online / continual / continuous / incremental / lifelong learning
  
  Further, some familiar branches of ML (like Transfer Learning and Multi-task learning) have a lot of intersection with Continual Learning.
  
  ---
  
  Related q's:
  
  * [What is the difference between active learning and online learning?](https://ai.stackexchange.com/questions/23226/what-is-the-difference-between-active-learning-and-online-learning)
  * <https://stats.stackexchange.com/questions/897/online-vs-offline-learning>
  
  
  ---
  
  