Answers.Body: <p>I think the squared image is more a choice for simplicity.</p>
  
  <p>There are two types of convolutional neural networks</p>
  
  <ul>
  <li><strong>Traditional CNNs</strong>: CNNs that have fully connected layers at the end, and </li>
  <li><strong>fully convolutional networks (FCNs)</strong>: they are only made of convolutional layers (and subsampling and upsampling layers), so they do not contain fully connected layers</li>
  </ul>
  
  <p>With traditional CNNs, the inputs always need to have the same shape, because you flatten the last convolutional layer, with a fixed size. As the flatten layer has a fixed size, the feature map shape from the layer before has to be the same shape, and so the inputs (images) also have to.</p>
  
  <p>However, in FCN, you don't flatten the last convolutional layer, so you don't need a fixed feature map shape, and so you don't need an input with a fixed size.</p>
  
  <p>In both cases, you don't need a squared image. You just have to be careful in the case you use CNN with a fully connected layer, to have the right shape for the flatten layer. </p>
  
  <p>For instance, if you have an input size <span class="math-container">$320 \times 160$</span>, and you have 3 pooling layers, so your output in the last convolutional layer is <span class="math-container">$40 \times 20 \times c$</span> (with <span class="math-container">$c$</span> the number of filters/channels), then you just need the flatten layer to have <span class="math-container">$40*20*c$</span> neurons.</p>
  
  <p>If you create a new network, just design it to handle a rectangle image. </p>
  
  <p>If you want to use an already pre-trained one, I think the better choice is to resize the image. </p>
  
  <p>If the information in the cropped parts is important, maybe your prediction can be wrong (it depends if the object of interest is in the parts of the image that is cropped). Actually, in Yolo (an object recognition network), images are resized if they don't fit the input requirements. <a href="https://pjreddie.com/media/files/papers/yolo.pdf" rel="nofollow noreferrer">See figure 1 of the YOLO paper</a>. It's because you don't need a high resolution to detect an object (for example, the CIFAR dataset has images of shape <span class="math-container">$32 \times 32$</span>, but the network can still predict the correct label). So, I think that resizing your image may not affect the prediction much (unless the new size is very different from the original)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-09T13:07:45.250Z
Answers.LastActivityDate: 2020-06-14T10:49:25.380Z
Answers.LastEditDate: 2020-06-14T10:49:25.380Z
Answers.UserDisplayName: Jérémy Blain
Answers.Body: <p>If you have a rectangular image and you are using existing models (or existing code), then you have to add an input pre-processing pipeline which transforms the image to standard dimensions. This is very common in computer vision and both PyTorch and Tensorflow have support for easily adding input pre-processing input pipeline for such a transformation.</p>
  
  <p>Also, if you have a fixed size rectangular image data, then you can design your own network architecture (or initial module) which takes image features into account by using asymmetric pooling and convolutions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-10T04:28:03.833Z
Answers.LastActivityDate: 2020-06-14T10:52:26.097Z
Answers.LastEditDate: 2020-06-14T10:52:26.097Z
Answers.UserDisplayName: Abhishek Singh
Body: <p>Almost all the convolutional neural network architecture I have come across have a square input size of an image, like <span class="math-container">$32 \times 32$</span>, <span class="math-container">$64 \times 64$</span> or <span class="math-container">$128 \times 128$</span>. Ideally, we might not have a square image for all kinds of scenarios. For example, we could have an image of size <span class="math-container">$384 \times 256$</span></p>
  
  <p>My question is how to we handle such images during</p>
  
  <ol>
  <li>training,</li>
  <li>development, and</li>
  <li>testing</li>
  </ol>
  
  <p>of a neural network?</p>
  
  <p>Do we force the image to resize to the input of the neural network or just crop the image to the required input size?</p>
  
Tags: <neural-networks><convolutional-neural-networks><fully-convolutional-networks>
Title: How to handle rectangular images in convolutional neural networks?
UserDisplayName: Santhosh Dhaipule Chandrakanth
ThreadText: # How to handle rectangular images in convolutional neural networks?
  
  (Asked by: username_0 on 2018-10-09)
  
  Almost all the convolutional neural network architecture I have come across have a square input size of an image, like $32 \times 32$, $64 \times 64$ or $128 \times 128$. Ideally, we might not have a square image for all kinds of scenarios. For example, we could have an image of size $384 \times 256$
  
  My question is how to we handle such images during
  
  1. training,
  2. development, and
  3. testing
  
  of a neural network?
  
  Do we force the image to resize to the input of the neural network or just crop the image to the required input size?
  
  **Tags:** neural-networks, convolutional-neural-networks, fully-convolutional-networks
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-10-09. Score: 9
  
  I think the squared image is more a choice for simplicity.
  
  There are two types of convolutional neural networks
  
  * **Traditional CNNs**: CNNs that have fully connected layers at the end, and
  * **fully convolutional networks (FCNs)**: they are only made of convolutional layers (and subsampling and upsampling layers), so they do not contain fully connected layers
  
  With traditional CNNs, the inputs always need to have the same shape, because you flatten the last convolutional layer, with a fixed size. As the flatten layer has a fixed size, the feature map shape from the layer before has to be the same shape, and so the inputs (images) also have to.
  
  However, in FCN, you don't flatten the last convolutional layer, so you don't need a fixed feature map shape, and so you don't need an input with a fixed size.
  
  In both cases, you don't need a squared image. You just have to be careful in the case you use CNN with a fully connected layer, to have the right shape for the flatten layer.
  
  For instance, if you have an input size $320 \times 160$, and you have 3 pooling layers, so your output in the last convolutional layer is $40 \times 20 \times c$ (with $c$ the number of filters/channels), then you just need the flatten layer to have $40\*20\*c$ neurons.
  
  If you create a new network, just design it to handle a rectangle image.
  
  If you want to use an already pre-trained one, I think the better choice is to resize the image.
  
  If the information in the cropped parts is important, maybe your prediction can be wrong (it depends if the object of interest is in the parts of the image that is cropped). Actually, in Yolo (an object recognition network), images are resized if they don't fit the input requirements. [See figure 1 of the YOLO paper](https://pjreddie.com/media/files/papers/yolo.pdf). It's because you don't need a high resolution to detect an object (for example, the CIFAR dataset has images of shape $32 \times 32$, but the network can still predict the correct label). So, I think that resizing your image may not affect the prediction much (unless the new size is very different from the original)
  
  
  ---
  
  ### Answer by username_2 on 2018-10-10. Score: 1
  
  If you have a rectangular image and you are using existing models (or existing code), then you have to add an input pre-processing pipeline which transforms the image to standard dimensions. This is very common in computer vision and both PyTorch and Tensorflow have support for easily adding input pre-processing input pipeline for such a transformation.
  
  Also, if you have a fixed size rectangular image data, then you can design your own network architecture (or initial module) which takes image features into account by using asymmetric pooling and convolutions.
  
  
  ---
  
  