Answers.Body: <p>(Of course, similar questions have been asked in the past and there are many sites, papers, video lessons, online that explain how CNNs work, but I think it's still a good idea to have a reference answer that hopefully will give you the main ideas behind CNNs.)</p>
  <p>A convolutional neural network (CNN) is a neural network that performs the <strong>convolution</strong> (or cross-correlation) operation typically followed by some <strong>downsampling</strong> (aka pooling) operations.</p>
  <p>The convolution operation comes from the mathematical equivalent operation, which is an operation that takes as inputs two functions <span class="math-container">$f$</span> and <span class="math-container">$h$</span> and produces another function <span class="math-container">$g$</span> as output, which is often denoted as <span class="math-container">$f \circledast h = g$</span>, where <span class="math-container">$\circledast$</span> is the convolution operation (or operator).</p>
  <p>In image processing, the convolution is used to process images in multiples ways. For example, it is used to remove noise (e.g. the convolution of a noisy image with a Gaussian kernel produces a smoother image) or to compute derivatives of the image, which can then be used e.g. to detect edges or corners, which are usually the main features of an image. For example, the <a href="http://www.bmva.org/bmvc/1988/avc-88-023.pdf" rel="nofollow noreferrer"><em>Harris corner detector</em></a> makes use of the partial derivatives (in the <span class="math-container">$x$</span> and <span class="math-container">$y$</span> direction) of the image to find corners (or interest points) in the image.</p>
  <p>In the context of CNNs, <span class="math-container">$f$</span> is the <strong>image</strong> (in the case of the first layer of the CNN) or a so-called <em>feature map</em> (in the case of hidden layers), <span class="math-container">$h$</span> is the <strong>kernel</strong> (aka <em>filter</em>) and <span class="math-container">$g$</span> is also a <strong>feature map</strong>. (In <a href="https://ai.stackexchange.com/a/21401/2444">this answer</a>, I explain these concepts, including how an image can be viewed as a function, more in detail, so I suggest that you read it).</p>
  <p>Here's an illustrative example of how the convolution works.</p>
  <p><a href="https://i.sstatic.net/8fjfx.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8fjfx.gif" alt="enter image description here" /></a></p>
  <p>where the <span class="math-container">$\color{blue}{\text{blue}}$</span> grid is the image, the <span class="math-container">$\color{gray}{\text{gray}}$</span> grid is the kernel and <span class="math-container">$\color{green}{\text{green}}$</span> grid is the feature map (i.e. the output of the convolution between the image and the kernel). The white squares are around the images represent the padding that is added around the image so that the convolution operation produces a feature map of a specific dimension.</p>
  <p>Essentially, the convolution is <strong>a series of dot products</strong> between the kernel and different patches (or parts) of the image. <a href="https://ai.stackexchange.com/q/11172/2444">The convolution can even be represented as matrix multiplication</a>, so the name <em>convolution</em> shouldn't scare you anymore, if you are familiar with dot products and matrix multiplications.</p>
  <p>As opposed to many operations in image processing where the kernels are typically fixed, in the context of CNNs, the kernels are <strong>learnable</strong> parameters, i.e. they change depending on the loss function, so they are supposed to represent functions that when convolved with their respective input functions are useful to extract meaningful information (i.e. features) to solve the task the CNN is being trained to solve. For this reason, the convolution is often thought of as an operation that extracts features from images. In fact, the output of the convolution, in the context of CNNs, is often called <em>feature map</em>. Moreover, a CNN is typically thought of as a <strong>data-driven feature extractor</strong> for the same reason.</p>
  <p>There are many different variations of the standard convolution operation. For example, there are <strong>transposed convolutions</strong>, <strong>dilated convolutions</strong>, etc., which are used to solve slightly different problems. For example, the dilated convolution can be used to increase the receptive field of an element of a feature map (which is typically a desirable property for several reasons). There are also <strong>upsampling</strong> operations which are particularly useful in the context of image segmentation. These different convolutions and their arithmetic are explained very well in the paper <a href="https://arxiv.org/pdf/1603.07285.pdf" rel="nofollow noreferrer">A guide to convolution arithmetic for deep learning</a> by Vincent Dumoulin and Francesco Visin. <a href="https://github.com/vdumoulin/conv_arithmetic" rel="nofollow noreferrer">Here</a>'s also the associated Github repository that contains all the images in this paper that show how the convolution operations work (from which I also took the gif above).</p>
  <p>To conclude, CNNs are very useful to process images and extract features from them because they use convolution operations (and downsampling and upsampling operations). They can be used for <strong>image</strong> (or object) <strong>classification</strong>, <strong>object detection</strong> (i.e. object localization with a bounding box + object classification), <strong>image segmentation</strong> (including semantic segmentation and instance segmentation), and possibly many other tasks where you need to learn a function that takes as input images and needs to extract information from those images to get someone high-level (but also low-level) output (e.g. the name of the object in the image), given some training data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-17T12:28:09.867Z
Answers.LastActivityDate: 2020-06-17T12:55:33.060Z
Answers.LastEditDate: 2020-06-17T12:55:33.060Z
Answers.UserDisplayName: nbro
Body: <p>Given that this question has not yet been asked on this site, although similar questions have already been asked in the past (e.g. <a href="https://ai.stackexchange.com/q/5546/2444">here</a> or <a href="https://ai.stackexchange.com/q/21394/2444">here</a>), what is essentially a convolutional neural network (CNN)? Why are they heavily used in image recognition applications of machine learning?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-17T04:49:08.993Z
Comments.Text: I'm not sure this is the best place to ask as broad a question as this. However, I would recommend watching this lecture if you want some insight: [Lecture 1 | Introduction to Convolutional Neural Networks for Visual Recognition](https://www.youtube.com/watch?v=vT1JzLTH4G4&list=PLC1qU-LWwrF64f4QKQT-Vg5Wr4qEE1Zxk&index=2&t=0s) by Standford.
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-17T05:04:39.063Z
Comments.Text: I can't recommend this series of blog posts enough: [A Beginner's Guide To Understanding Convolutional Neural Networks](https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/). They've helped me a lot to understand CNNs back in the day when I was studying them.
Comments.UserDisplayName: SpiderRico
Tags: <neural-networks><convolutional-neural-networks><computer-vision><definitions><image-processing>
Title: What is a convolutional neural network?
UserDisplayName: Pluviophile
ThreadText: # What is a convolutional neural network?
  
  (Asked by: username_0 on 2020-06-17)
  
  Given that this question has not yet been asked on this site, although similar questions have already been asked in the past (e.g. [here](https://ai.stackexchange.com/q/5546/2444) or [here](https://ai.stackexchange.com/q/21394/2444)), what is essentially a convolutional neural network (CNN)? Why are they heavily used in image recognition applications of machine learning?
  
  **Tags:** neural-networks, convolutional-neural-networks, computer-vision, definitions, image-processing
  
  > **Comments:**
  > **username_1**: I'm not sure this is the best place to ask as broad a question as this. However, I would recommend watching this lecture if you want some insight: [Lecture 1 | Introduction to Convolutional Neural Networks for Visual Recognition](https://www.youtube.com/watch?v=vT1JzLTH4G4&list=PLC1qU-LWwrF64f4QKQT-Vg5Wr4qEE1Zxk&index=2&t=0s) by Standford.  
  > **username_2**: I can't recommend this series of blog posts enough: [A Beginner's Guide To Understanding Convolutional Neural Networks](https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/). They've helped me a lot to understand CNNs back in the day when I was studying them.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-06-17. Score: 4 | Accepted answer
  
  (Of course, similar questions have been asked in the past and there are many sites, papers, video lessons, online that explain how CNNs work, but I think it's still a good idea to have a reference answer that hopefully will give you the main ideas behind CNNs.)
  
  A convolutional neural network (CNN) is a neural network that performs the **convolution** (or cross-correlation) operation typically followed by some **downsampling** (aka pooling) operations.
  
  The convolution operation comes from the mathematical equivalent operation, which is an operation that takes as inputs two functions $f$ and $h$ and produces another function $g$ as output, which is often denoted as $f \circledast h = g$, where $\circledast$ is the convolution operation (or operator).
  
  In image processing, the convolution is used to process images in multiples ways. For example, it is used to remove noise (e.g. the convolution of a noisy image with a Gaussian kernel produces a smoother image) or to compute derivatives of the image, which can then be used e.g. to detect edges or corners, which are usually the main features of an image. For example, the [*Harris corner detector*](http://www.bmva.org/bmvc/1988/avc-88-023.pdf) makes use of the partial derivatives (in the $x$ and $y$ direction) of the image to find corners (or interest points) in the image.
  
  In the context of CNNs, $f$ is the **image** (in the case of the first layer of the CNN) or a so-called *feature map* (in the case of hidden layers), $h$ is the **kernel** (aka *filter*) and $g$ is also a **feature map**. (In [this answer](https://ai.stackexchange.com/a/21401/2444), I explain these concepts, including how an image can be viewed as a function, more in detail, so I suggest that you read it).
  
  Here's an illustrative example of how the convolution works.
  
  [![enter image description here](https://i.sstatic.net/8fjfx.gif)](https://i.sstatic.net/8fjfx.gif)
  
  where the $\color{blue}{\text{blue}}$ grid is the image, the $\color{gray}{\text{gray}}$ grid is the kernel and $\color{green}{\text{green}}$ grid is the feature map (i.e. the output of the convolution between the image and the kernel). The white squares are around the images represent the padding that is added around the image so that the convolution operation produces a feature map of a specific dimension.
  
  Essentially, the convolution is **a series of dot products** between the kernel and different patches (or parts) of the image. [The convolution can even be represented as matrix multiplication](https://ai.stackexchange.com/q/11172/2444), so the name *convolution* shouldn't scare you anymore, if you are familiar with dot products and matrix multiplications.
  
  As opposed to many operations in image processing where the kernels are typically fixed, in the context of CNNs, the kernels are **learnable** parameters, i.e. they change depending on the loss function, so they are supposed to represent functions that when convolved with their respective input functions are useful to extract meaningful information (i.e. features) to solve the task the CNN is being trained to solve. For this reason, the convolution is often thought of as an operation that extracts features from images. In fact, the output of the convolution, in the context of CNNs, is often called *feature map*. Moreover, a CNN is typically thought of as a **data-driven feature extractor** for the same reason.
  
  There are many different variations of the standard convolution operation. For example, there are **transposed convolutions**, **dilated convolutions**, etc., which are used to solve slightly different problems. For example, the dilated convolution can be used to increase the receptive field of an element of a feature map (which is typically a desirable property for several reasons). There are also **upsampling** operations which are particularly useful in the context of image segmentation. These different convolutions and their arithmetic are explained very well in the paper [A guide to convolution arithmetic for deep learning](https://arxiv.org/pdf/1603.07285.pdf) by Vincent Dumoulin and Francesco Visin. [Here](https://github.com/vdumoulin/conv_arithmetic)'s also the associated Github repository that contains all the images in this paper that show how the convolution operations work (from which I also took the gif above).
  
  To conclude, CNNs are very useful to process images and extract features from them because they use convolution operations (and downsampling and upsampling operations). They can be used for **image** (or object) **classification**, **object detection** (i.e. object localization with a bounding box + object classification), **image segmentation** (including semantic segmentation and instance segmentation), and possibly many other tasks where you need to learn a function that takes as input images and needs to extract information from those images to get someone high-level (but also low-level) output (e.g. the name of the object in the image), given some training data.
  
  
  ---
  
  