Answers.Body: <p>If you have a <span class="math-container">$h_i \times w_i \times d_i$</span> input, where <span class="math-container">$h_i, w_i$</span> and <span class="math-container">$d_i$</span> respectively refer to the height, width and depth of the input, then we usually apply <span class="math-container">$m$</span> <span class="math-container">$h_k \times w_k \times d_i$</span> kernels (or filters) to this input (with the appropriate stride and padding), where <span class="math-container">$m$</span> is usually a <a href="https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)" rel="nofollow noreferrer">hyper-parameter</a>. So, after the application of <span class="math-container">$m$</span> kernels, you will obtain <span class="math-container">$m$</span> <span class="math-container">$h_o \times w_o \times 1$</span> so-called <em>feature maps</em> (also known as <em>activation maps</em>), which are usually concatenated along the depth dimension, hence your output will have a depth of <span class="math-container">$m$</span> (given that the application of a kernel to the input usually produces a two-dimensional output). For this reason, the output is usually referred to as <em>output volume</em>.</p>
  
  <p>In the context of CNNs, the kernels are learned, so they are not constant (at least, during the learning process, but, after training, they usually remain constant, unless you perform continual lifelong learning). Each kernel will be different from any other kernel, so each kernel will be doing a different convolution with the input (with respect to the other kernels), therefore, each kernel will be responsible for filtering (or detecting) a specific and different (with respect to the other kernels) feature of the input, which can, for example, be the initial image or the output of another convolutional layer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T08:31:34.373Z
Answers.LastActivityDate: 2019-08-16T08:38:35.733Z
Answers.LastEditDate: 2019-08-16T08:38:35.733Z
Answers.UserDisplayName: nbro
Body: <p>In a convolutional neural network, when we apply the convolution on a <span class="math-container">$5 \times 5$</span> image with <span class="math-container">$3 \times 3$</span> kernel, with stride <span class="math-container">$1$</span>, we should get only one <span class="math-container">$4 \times 4$</span> as output. In most of the CNN tutorials, we are having <span class="math-container">$4 \times 4 \times m$</span> as output. I don't know how we are getting a three-dimensional output and I don't know how we need to calculate <span class="math-container">$m$</span>. How is <span class="math-container">$m$</span> determined? Why do we get a three-dimensional output after a convolutional layer?</p>
  
Tags: <convolutional-neural-networks><computer-vision><image-processing><convolution>
Title: Why do we get a three-dimensional output after a convolutional layer?
UserDisplayName: Prabu M
ThreadText: # Why do we get a three-dimensional output after a convolutional layer?
  
  (Asked by: username_0 on 2019-08-16)
  
  In a convolutional neural network, when we apply the convolution on a $5 \times 5$ image with $3 \times 3$ kernel, with stride $1$, we should get only one $4 \times 4$ as output. In most of the CNN tutorials, we are having $4 \times 4 \times m$ as output. I don't know how we are getting a three-dimensional output and I don't know how we need to calculate $m$. How is $m$ determined? Why do we get a three-dimensional output after a convolutional layer?
  
  **Tags:** convolutional-neural-networks, computer-vision, image-processing, convolution
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-08-16. Score: 2 | Accepted answer
  
  If you have a $h\_i \times w\_i \times d\_i$ input, where $h\_i, w\_i$ and $d\_i$ respectively refer to the height, width and depth of the input, then we usually apply $m$ $h\_k \times w\_k \times d\_i$ kernels (or filters) to this input (with the appropriate stride and padding), where $m$ is usually a [hyper-parameter](https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)). So, after the application of $m$ kernels, you will obtain $m$ $h\_o \times w\_o \times 1$ so-called *feature maps* (also known as *activation maps*), which are usually concatenated along the depth dimension, hence your output will have a depth of $m$ (given that the application of a kernel to the input usually produces a two-dimensional output). For this reason, the output is usually referred to as *output volume*.
  
  In the context of CNNs, the kernels are learned, so they are not constant (at least, during the learning process, but, after training, they usually remain constant, unless you perform continual lifelong learning). Each kernel will be different from any other kernel, so each kernel will be doing a different convolution with the input (with respect to the other kernels), therefore, each kernel will be responsible for filtering (or detecting) a specific and different (with respect to the other kernels) feature of the input, which can, for example, be the initial image or the output of another convolutional layer.
  
  
  ---
  
  