Answers.Body: <p><a href="https://arxiv.org/pdf/1411.4038.pdf" rel="nofollow noreferrer">Fully convolutional networks</a> (FCNs) replace dense layers with <span class="math-container">$1\times 1$</span> convolutional layers in order to handle different input sizes: as <span class="math-container">$K\times K$</span> kernels are applied to spatial locations that can vary in number, instead a dense weight matrix has a fixed shape that can't adapt to a different input dimensionality.</p>
  <p>Now, <em>what about an input image that is smaller than the ones used for training?</em></p>
  <p>I think the elements to consider are the following: <em>kernel size</em>, <em>padding</em>, <em>strides</em> and <em>pooling</em> layers. Considering a fixed kernel size <span class="math-container">$K$</span> for each convolutional layer, each time we apply a conv the activation volume will be smaller in size (width and height) unless padding is used. The same is true for strided convolutions and pooling layers: <strong>they shrink the output by a given factor</strong> (e.g. <span class="math-container">$2$</span>).</p>
  <p>Suppose there are <span class="math-container">$L$</span> conv layers (without padding), and <span class="math-container">$P$</span> pooling layers with stride <span class="math-container">$2$</span>, the <strong>total stride</strong> of the model is: <span class="math-container">$S=(K\times L) + 2^P$</span>. This means that, in principle, <span class="math-container">$H\times W$</span> input <em><strong>images should be larger than the total stride</strong></em>, i.e. <span class="math-container">$H\ge S+1$</span> and <span class="math-container">$W\ge S+1$</span> (indeed, considering appropriate rounding to the integer part.)</p>
  <p>So the penultimate activation map should be at least <span class="math-container">$1\times 1$</span> in size in order to be processed by the last point-wise convolution that makes dense predictions.</p>
  <p><strong>NOTE:</strong> I haven't verified this in practice, but I want to provide some intuitions and these are indeed practical constraints in order for convolutions or pooling to work at all: <em>you cannot slide a kernel on an image that is smaller than the kernel itself</em>. So, extending this to multiple layers you obtain <span class="math-container">$S$</span> (that I call the total stride, but I don't know if there is a proper name for this.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-01T17:37:22.573Z
Answers.Text: Thanks for the response. I don't think FCN replaces dense layers with 1x1 convolutions where the kernel size is $1\times1\times D$. Instead, it replaces dense layers with kernels that have the same dimension as its input (dim of the previous layer) during training, such that the output of such layer is $1\times1\times N$. This means the previous layer would have to be as large as the kernel. Maybe the input image that are too small will have to be padded or resized during testing?
Answers.UserDisplayName: Tom Bennett
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-01T20:16:27.830Z
Answers.Text: Sure, padding or resizing will do the job. Probably for very small images you can up-sample them, forward the model, then down-sample the segmentation masks and apply them to the original image.
Answers.UserDisplayName: Luca Anzalone
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-02T00:03:32.820Z
Answers.Text: It feels odd that I haven't seen this problem mentioned elsewhere.
Answers.UserDisplayName: Tom Bennett
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-01T15:26:57.333Z
Answers.LastActivityDate: 2023-05-01T15:26:57.333Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>It is said that a fully convolutional network can handle any image size. I don't understand.</p>
  <p>Unlike regular CNN, a fully convolutional network reinterprets the dense layer as a convolutional operation. Specifically, we think of it as a list of kernels, each having the same size as the input (a.k.a. the activations of the previous layer) during training. This is fine if the input image at test time is larger, because the input size will be bigger, and the kernel can slide around. But it seems a problem if the image is smaller, as the input size will be smaller than the kernel.</p>
  <p>So I wonder how a fully convolutional network can handle smaller images.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-01T12:57:18.307Z
Comments.Text: Do these posts [1](https://ai.stackexchange.com/questions/13082/why-can-a-fully-convolutional-network-accept-images-of-any-size?rq=1) and [2](https://ai.stackexchange.com/questions/21810/what-is-a-fully-convolution-network?rq=1) answer your question?
Comments.UserDisplayName: Luca Anzalone
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-01T13:29:36.940Z
Comments.Text: No, I had read both before I wrote this question. I also read the original paper.
Comments.UserDisplayName: Tom Bennett
Tags: <fully-convolutional-networks>
Title: Can a fully convolutional network handle smaller images at test time?
UserDisplayName: Tom Bennett
ThreadText: # Can a fully convolutional network handle smaller images at test time?
  
  (Asked by: username_0 on 2023-04-30)
  
  It is said that a fully convolutional network can handle any image size. I don't understand.
  
  Unlike regular CNN, a fully convolutional network reinterprets the dense layer as a convolutional operation. Specifically, we think of it as a list of kernels, each having the same size as the input (a.k.a. the activations of the previous layer) during training. This is fine if the input image at test time is larger, because the input size will be bigger, and the kernel can slide around. But it seems a problem if the image is smaller, as the input size will be smaller than the kernel.
  
  So I wonder how a fully convolutional network can handle smaller images.
  
  **Tags:** fully-convolutional-networks
  
  > **Comments:**
  > **username_1**: Do these posts [1](https://ai.stackexchange.com/questions/13082/why-can-a-fully-convolutional-network-accept-images-of-any-size?rq=1) and [2](https://ai.stackexchange.com/questions/21810/what-is-a-fully-convolution-network?rq=1) answer your question?  
  > **username_0**: No, I had read both before I wrote this question. I also read the original paper.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-05-01. Score: 0
  
  [Fully convolutional networks](https://arxiv.org/pdf/1411.4038.pdf) (FCNs) replace dense layers with $1\times 1$ convolutional layers in order to handle different input sizes: as $K\times K$ kernels are applied to spatial locations that can vary in number, instead a dense weight matrix has a fixed shape that can't adapt to a different input dimensionality.
  
  Now, *what about an input image that is smaller than the ones used for training?*
  
  I think the elements to consider are the following: *kernel size*, *padding*, *strides* and *pooling* layers. Considering a fixed kernel size $K$ for each convolutional layer, each time we apply a conv the activation volume will be smaller in size (width and height) unless padding is used. The same is true for strided convolutions and pooling layers: **they shrink the output by a given factor** (e.g. $2$).
  
  Suppose there are $L$ conv layers (without padding), and $P$ pooling layers with stride $2$, the **total stride** of the model is: $S=(K\times L) + 2^P$. This means that, in principle, $H\times W$ input ***images should be larger than the total stride***, i.e. $H\ge S+1$ and $W\ge S+1$ (indeed, considering appropriate rounding to the integer part.)
  
  So the penultimate activation map should be at least $1\times 1$ in size in order to be processed by the last point-wise convolution that makes dense predictions.
  
  **NOTE:** I haven't verified this in practice, but I want to provide some intuitions and these are indeed practical constraints in order for convolutions or pooling to work at all: *you cannot slide a kernel on an image that is smaller than the kernel itself*. So, extending this to multiple layers you obtain $S$ (that I call the total stride, but I don't know if there is a proper name for this.)
  
  > **Comments:**
  > **username_0**: Thanks for the response. I don't think FCN replaces dense layers with 1x1 convolutions where the kernel size is $1\times1\times D$. Instead, it replaces dense layers with kernels that have the same dimension as its input (dim of the previous layer) during training, such that the output of such layer is $1\times1\times N$. This means the previous layer would have to be as large as the kernel. Maybe the input image that are too small will have to be padded or resized during testing?  
  > **username_1**: Sure, padding or resizing will do the job. Probably for very small images you can up-sample them, forward the model, then down-sample the segmentation masks and apply them to the original image.  
  > **username_0**: It feels odd that I haven't seen this problem mentioned elsewhere.  
  
  ---
  
  