Answers.Body: <blockquote>
  <p>If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why?</p>
  </blockquote>
  <p>If the convolutional neural network (CNN) <strong>only</strong> uses convolutional layers, then the number of parameters does not increase as a function of the <strong>spatial</strong> dimensions (<span class="math-container">$x$</span> and <span class="math-container">$y$</span>) of the input. This is one of the advantages of CNNs!</p>
  <p>The reason is quite simple: the parameters of the convolutional layers are the <em>kernels</em> (aka filters), which typically have fixed dimensions and can, nevertheless, be applied to inputs of different <strong>spatial</strong> dimensions, provided that the necessary <em>padding</em> is used. However, note that padding can create bigger <em>feature maps</em>, but feature maps are not the parameters of the neural network: they are the output of the convolutional layers. That's probably what confuses you, when you see a diagram of a CNN, because you see bigger feature maps and you might think that the number of parameters increases.</p>
  <p>Here's a TensorFlow 2 (and Keras) program that shows that the number of parameters does not change as a function of the <span class="math-container">$x$</span> and <span class="math-container">$y$</span> dimension of the input.</p>
  <pre><code>import tensorflow as tf
  
  input_shapes = [(2 * k, 2 * k, 3) for k in range(2, 6)]
  
  for input_shape in input_shapes:
      model = tf.keras.Sequential()
      model.add(tf.keras.layers.Input(shape=input_shape))
      model.add(tf.keras.layers.Conv2D(10, kernel_size=3, use_bias=True))
      model.summary() # The total number of parameters is always 280
  </code></pre>
  <p>The parameters of a convolutional layer can increase if you increase the size of each kernel and the number of kernels, but this does not necessarily depend on the input.</p>
  <p>The parameters of the CNN can also increase if you increase the depth of the input, but that's typically fixed (either <span class="math-container">$3$</span> for RGB images or <span class="math-container">$1$</span> for grayscale images). The reason is quite simple too: the kernels in the first convolutional layer (connected to the input layer) will have the same depth as the depth of the input.</p>
  <p>If your CNN also has fully connected layers, then the number of parameters also depends on the dimensions of the inputs. This is because the parameters of the fully connected layers depend on the number and dimensions of the feature maps (remember the <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten?version=nightly" rel="nofollow noreferrer">flatten</a> layer before the fully connected layers?), which, as I said, can increase as a function of the input.</p>
  <p>If you don't want to use fully connected layers, you may want to try <a href="https://arxiv.org/pdf/1411.4038.pdf" rel="nofollow noreferrer"><strong>fully convolutional networks</strong></a> (FCNs), which do not make use of fully connected layers, but can, nonetheless, be used to solve classification (and other) tasks.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-22T20:23:26.313Z
Answers.LastActivityDate: 2020-06-23T12:02:21.457Z
Answers.LastEditDate: 2020-06-23T12:02:21.457Z
Answers.UserDisplayName: nbro
Body: <p>If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the sizes and lengths of the filters are still the same, how can the number of parameter in a network increase?</p>
  
Tags: <convolutional-neural-networks><filters><weights>
Title: Does the number of parameters in a convolutional neuronal network increase if the input dimension increases?
UserDisplayName: Khan
ThreadText: # Does the number of parameters in a convolutional neuronal network increase if the input dimension increases?
  
  (Asked by: username_0 on 2020-06-22)
  
  If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the sizes and lengths of the filters are still the same, how can the number of parameter in a network increase?
  
  **Tags:** convolutional-neural-networks, filters, weights
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-22. Score: 1 | Accepted answer
  
  If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why?
  
  If the convolutional neural network (CNN) only uses convolutional layers, then the number of parameters does not increase as a function of the spatial dimensions ($x$ and $y$) of the input. This is one of the advantages of CNNs!
  The reason is quite simple: the parameters of the convolutional layers are the kernels (aka filters), which typically have fixed dimensions and can, nevertheless, be applied to inputs of different spatial dimensions, provided that the necessary padding is used. However, note that padding can create bigger feature maps, but feature maps are not the parameters of the neural network: they are the output of the convolutional layers. That's probably what confuses you, when you see a diagram of a CNN, because you see bigger feature maps and you might think that the number of parameters increases.
  Here's a TensorFlow 2 (and Keras) program that shows that the number of parameters does not change as a function of the $x$ and $y$ dimension of the input.
  import tensorflow as tf
  
  input_shapes = [(2 * k, 2 * k, 3) for k in range(2, 6)]
  
  for input_shape in input_shapes:
      model = tf.keras.Sequential()
      model.add(tf.keras.layers.Input(shape=input_shape))
      model.add(tf.keras.layers.Conv2D(10, kernel_size=3, use_bias=True))
      model.summary() # The total number of parameters is always 280
  
  The parameters of a convolutional layer can increase if you increase the size of each kernel and the number of kernels, but this does not necessarily depend on the input.
  The parameters of the CNN can also increase if you increase the depth of the input, but that's typically fixed (either $3$ for RGB images or $1$ for grayscale images). The reason is quite simple too: the kernels in the first convolutional layer (connected to the input layer) will have the same depth as the depth of the input.
  If your CNN also has fully connected layers, then the number of parameters also depends on the dimensions of the inputs. This is because the parameters of the fully connected layers depend on the number and dimensions of the feature maps (remember the flatten layer before the fully connected layers?), which, as I said, can increase as a function of the input.
  If you don't want to use fully connected layers, you may want to try fully convolutional networks (FCNs), which do not make use of fully connected layers, but can, nonetheless, be used to solve classification (and other) tasks.
  
  
  ---
  
  