Answers.Body: <p>Firstly, step size and padding have no influence on the number of parameters, they just determine where to apply your parameters, i.e. the convolution operator.</p>
  <blockquote>
  <p>My first thought was 32 because there are 32 neurons/filters [...].</p>
  </blockquote>
  <p>If you have 32 input channels and a kernel size of <span class="math-container">$3 \times 3$</span> then each entry of your <span class="math-container">$3\times3$</span>-filter acts on a 32-dimensional vector. Therefore the number of parameters of your kernel is <span class="math-container">$3 \cdot 3 \cdot 32$</span>. Or a more visual explanation: One convolutional filter squashes a <span class="math-container">$k \times k \times C^{in}$</span> <em>cube of data</em> into a single scalar, where each number in the cube has its own scalar in the kernel.</p>
  <p>If you want 64 output channels, you have to have 64 of these kernels, so the convolution you describe should have <span class="math-container">$3 \cdot 3 \cdot 32 \cdot 64$</span> parameters.</p>
  <p>As a generic formula: <span class="math-container">$\text{#Parameters}(\mathbf{W}) = k^h \cdot k^w \cdot C^{in} \cdot C^{out}$</span>. Here <span class="math-container">$k^h$</span> and <span class="math-container">$k^w$</span> denote kernel height and kernel width respectively, and <span class="math-container">$C^{in}$</span> and <span class="math-container">$C^{out}$</span> denote the number of in- and output channels.</p>
  <blockquote>
  <p>[...] and I thought that number of feature maps = number of filters</p>
  </blockquote>
  <p>That's right, but each filter <span class="math-container">$w$</span> is not actually a 2-dimensional window, but a 3-dimensional window, i.e. a rank 3 tensor <span class="math-container">$\mathbf{W} \in \mathbb{R}^{k \times k \times C^{in}}$</span>. You can see it in some visualizations of CNNs (s. image bloew). The input image has 3 channels and the first filter has 3-channels too and computes a single output pixel with only 1 channel.
  For further reading, <a href="https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53" rel="nofollow noreferrer">this</a> blog post seems to summarize convolution well.</p>
  <p><a href="https://i.sstatic.net/QGLzr.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QGLzr.png" alt="enter image description here" /></a>
  <em>Image taken from <a href="https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53" rel="nofollow noreferrer">here</a></em></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-03T09:40:37.370Z
Answers.Text: Ah thanks! Can I say 64 output channels = 64 neurons on this layer? Does that mean that each neuron have 3 * 3 * 32 * 64 connections?
Answers.UserDisplayName: Haidepzai
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-03T09:53:57.590Z
Answers.Text: Yes, 64 output channels correspond to 64 neurons. However, these neurons are not interconnected so one neuron has $3 \cdot 3 \cdot 32$ connections. Because you have 64 neurons, the number of parameters is $3 \cdot 3 \cdot 32 \cdot 64$.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-03T09:59:41.680Z
Answers.Text: If you think about it visually: The kernel is a pattern detector that detects a learned pattern of size $k \times k$ across all channels at once ($k \times k \times 32$) and outputs a scalar that describes how well the pattern matches. If you take 64 output channels your model has the capacity to learn 64 different patterns and produces an output scalar for each one - thus you get 64 output channels.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-03T12:35:26.703Z
Answers.Text: Thanks!! The number of parameters are the learnable parameters, right?
Answers.UserDisplayName: Haidepzai
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-03T14:01:34.673Z
Answers.Text: Jepp, you go it!
  Ah and just for completeness: A convolutional layer might also use a bias vector (by default in most deep learning libraries), in that case you'd get $3 \cdot 3 \cdot 32 \cdot 64 + 64$ learnable parameters. Because the bias is a 64-dimensional vector that is added to the final output before the activation function is applied.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-02T11:08:49.563Z
Answers.LastActivityDate: 2022-07-02T11:14:54.477Z
Answers.LastEditDate: 2022-07-02T11:14:54.477Z
Answers.UserDisplayName: Chillston
Body: <p>let's say I have a conv layer i with 64 feature maps and a filter size of 3x3.
  The previous conv layer i-1 has 32 feature map.
  Step-size is 2 and padding 1.</p>
  <p>My question is now how to know how many neurons of the previous layer are connected to each of the neuron in i?</p>
  <p>My first thought was 32 because there are 32 neurons/filters and I thought that number of feature maps = number of filters.</p>
  <p>But I am not sure</p>
  
Tags: <convolutional-neural-networks><filters>
Title: How to calculate number of connected neurons with filter
UserDisplayName: Haidepzai
ThreadText: # How to calculate number of connected neurons with filter
  
  (Asked by: username_0 on 2022-07-02)
  
  let's say I have a conv layer i with 64 feature maps and a filter size of 3x3.
  The previous conv layer i-1 has 32 feature map.
  Step-size is 2 and padding 1.
  
  My question is now how to know how many neurons of the previous layer are connected to each of the neuron in i?
  
  My first thought was 32 because there are 32 neurons/filters and I thought that number of feature maps = number of filters.
  
  But I am not sure
  
  **Tags:** convolutional-neural-networks, filters
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-02. Score: 2
  
  Firstly, step size and padding have no influence on the number of parameters, they just determine where to apply your parameters, i.e. the convolution operator.
  
  > My first thought was 32 because there are 32 neurons/filters [...].
  
  If you have 32 input channels and a kernel size of $3 \times 3$ then each entry of your $3\times3$-filter acts on a 32-dimensional vector. Therefore the number of parameters of your kernel is $3 \cdot 3 \cdot 32$. Or a more visual explanation: One convolutional filter squashes a $k \times k \times C^{in}$ *cube of data* into a single scalar, where each number in the cube has its own scalar in the kernel.
  
  If you want 64 output channels, you have to have 64 of these kernels, so the convolution you describe should have $3 \cdot 3 \cdot 32 \cdot 64$ parameters.
  
  As a generic formula: $\text{#Parameters}(\mathbf{W}) = k^h \cdot k^w \cdot C^{in} \cdot C^{out}$. Here $k^h$ and $k^w$ denote kernel height and kernel width respectively, and $C^{in}$ and $C^{out}$ denote the number of in- and output channels.
  
  > [...] and I thought that number of feature maps = number of filters
  
  That's right, but each filter $w$ is not actually a 2-dimensional window, but a 3-dimensional window, i.e. a rank 3 tensor $\mathbf{W} \in \mathbb{R}^{k \times k \times C^{in}}$. You can see it in some visualizations of CNNs (s. image bloew). The input image has 3 channels and the first filter has 3-channels too and computes a single output pixel with only 1 channel.
  For further reading, [this](https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53) blog post seems to summarize convolution well.
  
  [![enter image description here](https://i.sstatic.net/QGLzr.png)](https://i.sstatic.net/QGLzr.png)
  *Image taken from [here](https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53)*
  
  > **Comments:**
  > **username_0**: Ah thanks! Can I say 64 output channels = 64 neurons on this layer? Does that mean that each neuron have 3 * 3 * 32 * 64 connections?  
  > **username_1**: Yes, 64 output channels correspond to 64 neurons. However, these neurons are not interconnected so one neuron has $3 \cdot 3 \cdot 32$ connections. Because you have 64 neurons, the number of parameters is $3 \cdot 3 \cdot 32 \cdot 64$.  
  > **username_1**: If you think about it visually: The kernel is a pattern detector that detects a learned pattern of size $k \times k$ across all channels at once ($k \times k \times 32$) and outputs a scalar that describes how well the pattern matches. If you take 64 output channels your model has the capacity to learn 64 different patterns and produces an output scalar for each one - thus you get 64 output channels.  
  > **username_0**: Thanks!! The number of parameters are the learnable parameters, right?  
  > **username_1**: Jepp, you go it! Ah and just for completeness: A convolutional layer might also use a bias vector (by default in most deep learning libraries), in that case you'd get $3 \cdot 3 \cdot 32 \cdot 64 + 64$ learnable parameters. Because the bias is a 64-dimensional vector that is added to the final output before the activation function is applied.  
  
  ---
  
  