Answers.Body: <p>A convolutional layer is a layer where you slide a kernel or filter (which you can think of as a small square matrix of weights, which need to be learned during the learning phase) over the input. In practice, when you need to slide this kernel, you will often need to specify the "padding" (around the input) and "stride" (with which you convolve the kernel on the input), in order to obtain the desired output (size). So, even if you receive inputs of different sizes, you can change these values, like the padding or the stride, in order to produce a valid output (size). In this sense, I think, we can say that convolutional layers accept inputs of (almost) any size.</p>
  
  <p>The number of feature maps does not depend on the kernel or input (sizes). The number of features maps is determined by the number of different kernels that you will use to slide over the input. If you have <span class="math-container">$K$</span> different kernels, then you will have <span class="math-container">$K$</span> different feature maps. The number of kernels is often a hyper-parameter, so you can change it (as you please).</p>
  
  <p>A fully connected (FC) layer requires a fixed input size by design. The programmer decides the number of input units (or neurons) that the FC layer will have. This hyper-parameter often does not change during the learning phase. So, yes, FC often accept inputs of fixed size (also because they do not adopt techniques like "padding").</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-07T09:19:46.780Z
Answers.LastActivityDate: 2019-03-07T09:19:46.780Z
Answers.UserDisplayName: nbro
Answers.Body: <p>It doesn't have to be so. Fully connected layer could be considered as convolutional layer with input image of 1 pixel and spatial kernel size of 1 pixel. So 1-pixel kernel convolutional layer is effectively the same as fully connected layer attached to each pixel. That is idea behind "Fully Convolutional Networks". If you want "true", 1-pixel fully connected layer after convolutional layer (with variable input size) all you have to do is to put average pooling layer (or other type of pooling layer)  before fully connected layer. That way fully connected layer can accept variable input size.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-07T10:28:09.877Z
Answers.LastActivityDate: 2019-03-07T10:28:09.877Z
Answers.UserDisplayName: mirror2image
Body: <p>I'm studying how SPP (Spatial, Pyramid, Pooling) works. SPP was invented to tackle the fix input image size in CNN. According to the original paper <a href="https://arxiv.org/pdf/1406.4729.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1406.4729.pdf</a>, the authors say: </p>
  
  <blockquote>
    <p>convolutional layers do not require a fixed image size and can
    generate feature maps of any sizes. On the other hand, the
    fully-connected layers need to have fixed size/length input by their
    definition. Hence, the fixed size constraint comes only from the
    fully-connected layers, which exist at a deeper stage of the network.</p>
  </blockquote>
  
  <p>Why does a fully connected layer only accepts a fixed input size (but convolutional layers don't)? What's the real reason behind this definition?</p>
  
Tags: <deep-learning><convolutional-neural-networks><computer-vision>
Title: Why does a fully connected layer only accept a fixed input size?
UserDisplayName: iv67
ThreadText: # Why does a fully connected layer only accept a fixed input size?
  
  (Asked by: username_0 on 2019-03-07)
  
  I'm studying how SPP (Spatial, Pyramid, Pooling) works. SPP was invented to tackle the fix input image size in CNN. According to the original paper <https://arxiv.org/pdf/1406.4729.pdf>, the authors say:
  
  > convolutional layers do not require a fixed image size and can
  > generate feature maps of any sizes. On the other hand, the
  > fully-connected layers need to have fixed size/length input by their
  > definition. Hence, the fixed size constraint comes only from the
  > fully-connected layers, which exist at a deeper stage of the network.
  
  Why does a fully connected layer only accepts a fixed input size (but convolutional layers don't)? What's the real reason behind this definition?
  
  **Tags:** deep-learning, convolutional-neural-networks, computer-vision
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-03-07. Score: 1 | Accepted answer
  
  A convolutional layer is a layer where you slide a kernel or filter (which you can think of as a small square matrix of weights, which need to be learned during the learning phase) over the input. In practice, when you need to slide this kernel, you will often need to specify the "padding" (around the input) and "stride" (with which you convolve the kernel on the input), in order to obtain the desired output (size). So, even if you receive inputs of different sizes, you can change these values, like the padding or the stride, in order to produce a valid output (size). In this sense, I think, we can say that convolutional layers accept inputs of (almost) any size.
  
  The number of feature maps does not depend on the kernel or input (sizes). The number of features maps is determined by the number of different kernels that you will use to slide over the input. If you have $K$ different kernels, then you will have $K$ different feature maps. The number of kernels is often a hyper-parameter, so you can change it (as you please).
  
  A fully connected (FC) layer requires a fixed input size by design. The programmer decides the number of input units (or neurons) that the FC layer will have. This hyper-parameter often does not change during the learning phase. So, yes, FC often accept inputs of fixed size (also because they do not adopt techniques like "padding").
  
  
  ---
  
  ### Answer by username_2 on 2019-03-07. Score: 0
  
  It doesn't have to be so. Fully connected layer could be considered as convolutional layer with input image of 1 pixel and spatial kernel size of 1 pixel. So 1-pixel kernel convolutional layer is effectively the same as fully connected layer attached to each pixel. That is idea behind "Fully Convolutional Networks". If you want "true", 1-pixel fully connected layer after convolutional layer (with variable input size) all you have to do is to put average pooling layer (or other type of pooling layer) before fully connected layer. That way fully connected layer can accept variable input size.
  
  
  ---
  
  