Answers.Body: <p>It's inherited from math, but computer scientists optimized the algorithm and stuck with the original term.</p>
  <p>3Blue1Brown also comes to the same conclusion (<a href="https://youtu.be/KuXjwB4LzSA?t=804" rel="nofollow noreferrer">at 13:23</a>) that</p>
  <blockquote>
  <p>Another thing worth highlighting is that in the computer science context, this notion of flipping around that kernel before you let it march across the original often feels really weird and just uncalled for. But again, note that that's what's inherited from the pure math context, where like we saw with the probabilities, it's an incredibly natural thing to do.</p>
  </blockquote>
  <p>One of the first mentions of 'convolutions' in neural networks is in the seminal paper by <a href="https://proceedings.neurips.cc/paper/1989/file/53c3bce66e43be4f209556518c2fcb54-Paper.pdf" rel="nofollow noreferrer">Yann Le Cun et al (1989)</a>. However, they also do not specify whether they perform true convolution. They simply state</p>
  <blockquote>
  <p>This operation is equivalent to a convolution with a small size kernel, followed by a squashing function.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-09T18:21:58.820Z
Answers.LastActivityDate: 2023-05-09T18:21:58.820Z
Answers.UserDisplayName: Robin van Hoorn
Answers.Body: <p>Maths and calculus. I remember studying convolution 40 years ago in the Digital Signal Processing subject at the University.</p>
  <blockquote>
  <p>&quot;It was the 1760s when the Swiss mathematical genius Leonhard Euler
  (1707–1783) suffered complete blindness, but this illness did not
  prevent him from contributing to mathematics. In fact, in those years,
  he wrote a memorable book about integral calculus and the solution of
  differential equations (DEs) by certain definite integrals [6]. ....</p>
  <p>A first and simple case of this solution is K(u) = u and Q(x) = x and
  a second case is K(u) = -u and Q(x) = x. Euler studied the first case
  in [7], which is the first source in which the correlation operation
  is used exhaustively in statistical applications and related to
  convolution. &quot;</p>
  </blockquote>
  <p><a href="https://www.embs.org/pulse/articles/history-convolution-operation/" rel="nofollow noreferrer">A History of the Convolution Operation</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-27T13:55:04.710Z
Answers.LastActivityDate: 2023-05-27T13:55:04.710Z
Answers.UserDisplayName: Raul Alvarez
Body: <p>Convolutional neural networks (CNNs) contain convolutional layers. In modern deep learning libraries such as <a href="http://www.tensorflow.org" rel="nofollow noreferrer">Tensorflow</a> and <a href="http://www.pytorch.org" rel="nofollow noreferrer">PyTorch</a>, convolutional layers are implemented by using the cross-correlation operator instead of the convolution operator. The difference is that in convolution, the kernel is flipped before applying it to the input.</p>
  <p>For example, in the book <a href="https://www.deeplearningbook.org/" rel="nofollow noreferrer">&quot;Deep Learning&quot;</a>, it is explained as follows.</p>
  <blockquote>
  <p>Many machine learning libraries implement cross-correlation but call
  it convolution. --- In the
  context of machine learning, the learning algorithm will learn the
  appropriate values of the kernel in the appropriate place, so an
  algorithm based on convolution with kernel flipping will learn a
  kernel that is flipped relative to the kernel learned by an algorithm
  without the flipping. It is also rare for convolution to be used alone
  in machine learning; instead convolution is used simultaneously with
  other functions, and the combination of these functions does not
  commute regardless of whether the convolution operation flips its
  kernel or not.</p>
  </blockquote>
  <p>This makes perfect sense and convincingly argues why implementing the flipping of the kernel would be unnecessary.</p>
  <p>But how come CNNs are not commonly called &quot;cross-correlational neural networks&quot; instead of &quot;convolutional neural networks&quot;? To the best of my knowledge, the first concrete implementations of CNNs predate any of the above-mentioned libraries. Did these early implementations of CNNs indeed use the convolution operator, leading to the name? Or is there another reason?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-06T11:00:43.633Z
Comments.Text: [Here](https://ai.stackexchange.com/q/24921/2444) and [here](https://ai.stackexchange.com/q/21999/2444) are 2 questions that you may also be interested in.
Comments.UserDisplayName: nbro
Tags: <deep-learning><convolutional-neural-networks><terminology><history>
Title: Origins of the name of convolutional neural networks
UserDisplayName: mikkola
ThreadText: # Origins of the name of convolutional neural networks
  
  (Asked by: username_0 on 2021-05-04)
  
  Convolutional neural networks (CNNs) contain convolutional layers. In modern deep learning libraries such as [Tensorflow](http://www.tensorflow.org) and [PyTorch](http://www.pytorch.org), convolutional layers are implemented by using the cross-correlation operator instead of the convolution operator. The difference is that in convolution, the kernel is flipped before applying it to the input.
  
  For example, in the book ["Deep Learning"](https://www.deeplearningbook.org/), it is explained as follows.
  
  > Many machine learning libraries implement cross-correlation but call
  > it convolution. --- In the
  > context of machine learning, the learning algorithm will learn the
  > appropriate values of the kernel in the appropriate place, so an
  > algorithm based on convolution with kernel flipping will learn a
  > kernel that is flipped relative to the kernel learned by an algorithm
  > without the flipping. It is also rare for convolution to be used alone
  > in machine learning; instead convolution is used simultaneously with
  > other functions, and the combination of these functions does not
  > commute regardless of whether the convolution operation flips its
  > kernel or not.
  
  This makes perfect sense and convincingly argues why implementing the flipping of the kernel would be unnecessary.
  
  But how come CNNs are not commonly called "cross-correlational neural networks" instead of "convolutional neural networks"? To the best of my knowledge, the first concrete implementations of CNNs predate any of the above-mentioned libraries. Did these early implementations of CNNs indeed use the convolution operator, leading to the name? Or is there another reason?
  
  **Tags:** deep-learning, convolutional-neural-networks, terminology, history
  > **Comments:**
  > **username_1**: [Here](https://ai.stackexchange.com/q/24921/2444) and [here](https://ai.stackexchange.com/q/21999/2444) are 2 questions that you may also be interested in.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-05-09. Score: 1
  
  It's inherited from math, but computer scientists optimized the algorithm and stuck with the original term.
  
  3Blue1Brown also comes to the same conclusion ([at 13:23](https://youtu.be/KuXjwB4LzSA?t=804)) that
  
  > Another thing worth highlighting is that in the computer science context, this notion of flipping around that kernel before you let it march across the original often feels really weird and just uncalled for. But again, note that that's what's inherited from the pure math context, where like we saw with the probabilities, it's an incredibly natural thing to do.
  
  One of the first mentions of 'convolutions' in neural networks is in the seminal paper by [Yann Le Cun et al (1989)](https://proceedings.neurips.cc/paper/1989/file/53c3bce66e43be4f209556518c2fcb54-Paper.pdf). However, they also do not specify whether they perform true convolution. They simply state
  
  > This operation is equivalent to a convolution with a small size kernel, followed by a squashing function.
  
  
  ---
  
  ### Answer by username_3 on 2023-05-27. Score: 0
  
  Maths and calculus. I remember studying convolution 40 years ago in the Digital Signal Processing subject at the University.
  
  > "It was the 1760s when the Swiss mathematical genius Leonhard Euler
  > (1707–1783) suffered complete blindness, but this illness did not
  > prevent him from contributing to mathematics. In fact, in those years,
  > he wrote a memorable book about integral calculus and the solution of
  > differential equations (DEs) by certain definite integrals [6]. ....
  >
  > A first and simple case of this solution is K(u) = u and Q(x) = x and
  > a second case is K(u) = -u and Q(x) = x. Euler studied the first case
  > in [7], which is the first source in which the correlation operation
  > is used exhaustively in statistical applications and related to
  > convolution. "
  
  [A History of the Convolution Operation](https://www.embs.org/pulse/articles/history-convolution-operation/)
  
  
  ---
  
  