Answers.Body: <p>There is never a 100% accurate theory, however it's been observed to be beneficial, however I would argue that is due to the following:</p>
  <ul>
  <li>you want to have a latent dimension, to learn the manifold compressed version, so that the decoder has a harder life to overfit (kinda speculative)</li>
  <li>when doing convolution, you can keep the height and width dimensions constant, by introducing artifacts (which obviously you don't really want), so you are forced to decrease the dimensionality in H and W... so as a trade off, you just compensate with the number of filters/channels</li>
  </ul>
  <p>in other words, there is no correct answer, just empirical evidence that this &quot;latent compressed version&quot; works better in some cases</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-24T22:55:15.190Z
Answers.LastActivityDate: 2023-11-24T22:55:15.190Z
Answers.UserDisplayName: Alberto
Body: <p>When doing semantic segmentation, we often make use of FCN, which can be thought of in two parts: an encoder and decoder. As I understand, the encoder compresses the image into a spatially small, but high number of channels. The decoder then uses this high channel activation map and upsamples it into a representation detailing the class for each pixel in the image.</p>
  <p>My question is, why do we do this spatial compression at all?</p>
  <p>For example, why would the architecture shown below be a bad choice:</p>
  <p><a href="https://i.sstatic.net/MWQPQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/MWQPQ.png" alt="enter image description here" /></a></p>
  
Tags: <convolutional-neural-networks><semantic-segmentation><encoder-decoder><fully-convolutional-networks>
Title: Why do we do need compression in Semantic Segmentation?
UserDisplayName: Dude156
ThreadText: # Why do we do need compression in Semantic Segmentation?
  
  (Asked by: username_0 on 2023-11-24)
  
  When doing semantic segmentation, we often make use of FCN, which can be thought of in two parts: an encoder and decoder. As I understand, the encoder compresses the image into a spatially small, but high number of channels. The decoder then uses this high channel activation map and upsamples it into a representation detailing the class for each pixel in the image.
  
  My question is, why do we do this spatial compression at all?
  
  For example, why would the architecture shown below be a bad choice:
  
  [![enter image description here](https://i.sstatic.net/MWQPQ.png)](https://i.sstatic.net/MWQPQ.png)
  
  **Tags:** convolutional-neural-networks, semantic-segmentation, encoder-decoder, fully-convolutional-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-11-24. Score: 2 | Accepted answer
  
  There is never a 100% accurate theory, however it's been observed to be beneficial, however I would argue that is due to the following:
  
  * you want to have a latent dimension, to learn the manifold compressed version, so that the decoder has a harder life to overfit (kinda speculative)
  * when doing convolution, you can keep the height and width dimensions constant, by introducing artifacts (which obviously you don't really want), so you are forced to decrease the dimensionality in H and W... so as a trade off, you just compensate with the number of filters/channels
  
  in other words, there is no correct answer, just empirical evidence that this "latent compressed version" works better in some cases
  
  
  ---
  
  