Answers.Body: <p>A few years ago, deep learning was a buzzword, but now is de-facto a standard term or expression, and it's widely used in all research papers, although <em>deep learning</em> is almost never defined rigorously (but this doesn't seem to be a big problem!).</p>
  <p>From my experience (this is not just an opinion, of course!), after having read so many papers on the topic, deep learning typically refers to the <strong>subset of machine learning algorithms, specifically, gradient descent and back-propagation, applied to neural networks</strong>, and, in particular, neural networks with <strong>multiple layers</strong>. However, there is no consensus on what <em>multiple</em> refers to. In fact, Schmidhuber (co-author of the <a href="https://www.bioinf.jku.at/publications/older/2604.pdf" rel="nofollow noreferrer">LSTM</a>), in <a href="https://arxiv.org/pdf/1404.7828.pdf" rel="nofollow noreferrer">[1]</a>, writes</p>
  <blockquote>
  <p>At which problem depth does <em>Shallow Learning</em> end, and <em>Deep Learning</em> begin? Discussions with DL experts have not yet yielded a conclusive response to this question. Instead of committing myself to a precise answer, let me just define for the purposes of this overview: problems of depth &gt; 10 require <em>Very Deep Learning</em>.</p>
  </blockquote>
  <p>So, although there is typically no agreement on the definition of <em>deep</em> or <em>multiple</em>, there are several problems, such as the <strong>vanishing</strong> or <strong>exploding gradient problems</strong>, that arise as the number of layers in a neural network increases, so there are reasons behind the distinction of deep and non-deep/shallow learning.</p>
  <p>Note that some research papers (e.g., <a href="https://arxiv.org/pdf/1404.7828.pdf" rel="nofollow noreferrer">[1]</a>) distinguish between deep learning applied to neural networks and other types of deep learning, so as not to exclude the layered composition of other models (e.g., SVMs) other than neural networks and the use of machine learning algorithms (e.g. gradient descent) to train them.</p>
  <p>To answer your question more directly, <strong>neural networks are models</strong> or <strong>function approximators</strong>, while deep learning is the set of machine learning techniques applied to the layered composition of models (typically, neural networks).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-28T15:42:09.713Z
Answers.LastActivityDate: 2021-12-21T18:07:39.410Z
Answers.LastEditDate: 2021-12-21T18:07:39.410Z
Answers.UserDisplayName: nbro
Body: <p>I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers?</p>
  <p>What are formal definitions for these two?</p>
  
Tags: <neural-networks><deep-learning><comparison><definitions>
Title: What is the difference between artificial neural networks and deep learning?
UserDisplayName: pedrum
ThreadText: # What is the difference between artificial neural networks and deep learning?
  
  (Asked by: username_0 on 2020-07-28)
  
  I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers?
  
  What are formal definitions for these two?
  
  **Tags:** neural-networks, deep-learning, comparison, definitions
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-28. Score: 2
  
  A few years ago, deep learning was a buzzword, but now is de-facto a standard term or expression, and it's widely used in all research papers, although *deep learning* is almost never defined rigorously (but this doesn't seem to be a big problem!).
  
  From my experience (this is not just an opinion, of course!), after having read so many papers on the topic, deep learning typically refers to the **subset of machine learning algorithms, specifically, gradient descent and back-propagation, applied to neural networks**, and, in particular, neural networks with **multiple layers**. However, there is no consensus on what *multiple* refers to. In fact, Schmidhuber (co-author of the [LSTM](https://www.bioinf.jku.at/publications/older/2604.pdf)), in [[1]](https://arxiv.org/pdf/1404.7828.pdf), writes
  
  > At which problem depth does *Shallow Learning* end, and *Deep Learning* begin? Discussions with DL experts have not yet yielded a conclusive response to this question. Instead of committing myself to a precise answer, let me just define for the purposes of this overview: problems of depth > 10 require *Very Deep Learning*.
  
  So, although there is typically no agreement on the definition of *deep* or *multiple*, there are several problems, such as the **vanishing** or **exploding gradient problems**, that arise as the number of layers in a neural network increases, so there are reasons behind the distinction of deep and non-deep/shallow learning.
  
  Note that some research papers (e.g., [[1]](https://arxiv.org/pdf/1404.7828.pdf)) distinguish between deep learning applied to neural networks and other types of deep learning, so as not to exclude the layered composition of other models (e.g., SVMs) other than neural networks and the use of machine learning algorithms (e.g. gradient descent) to train them.
  
  To answer your question more directly, **neural networks are models** or **function approximators**, while deep learning is the set of machine learning techniques applied to the layered composition of models (typically, neural networks).
  
  
  ---
  
  