Answers.Body: <p>The point of DenseNet was to go as deep as ResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.</p>
  
  <p>About transition layers (convolution + pooling), I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from <span class="math-container">$56 \times 56$</span> to <span class="math-container">$28 \times 28$</span> to <span class="math-container">$14 \times 14$</span>, and so on. </p>
  
  <p>The authors state it this way</p>
  
  <blockquote>
    <p>To further improve model compactness, we can reduce the number of feature-maps at transition layers</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-15T08:37:11.813Z
Answers.Text: Thanks. So these transitions layers are here just to reduce complexity ? What is the price of this reduction ? Less precision ?
  
  If we had a really efficient computer, wouldn't it be better to have a single fully connected block ? So every feature map is accessible anywhere in the network ?
Answers.UserDisplayName: Astariul
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-15T08:45:58.093Z
Answers.Text: Theoretically seems yes, we should do better with fully connected, but we might have to find that out with implementation which doesn't look feasible practically! This is just my conjecture.
Answers.UserDisplayName: sagar_acharya
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-15T08:24:02.407Z
Answers.LastActivityDate: 2020-03-10T20:37:11.467Z
Answers.LastEditDate: 2020-03-10T20:37:11.467Z
Answers.UserDisplayName: sagar_acharya
Body: <p>The <a href="https://arxiv.org/pdf/1608.06993v3.pdf" rel="nofollow noreferrer">DenseNet architecture</a> can be summarizde with this figure:
  <a href="https://i.sstatic.net/cSwqp.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/cSwqp.png" alt="enter image description here"></a></p>
  
  <p><strong>Why there are transition layers between each block?</strong></p>
  
  <p>In the papers, they justify the use of transition layers as follow :</p>
  
  <blockquote>
    <p>The concatenation operation used in Eq. (2) is not viable when the size of feature-maps changes. However,  an essential part of convolutional networks is pooling layers that change the size of feature-maps. To facilitate pooling in our architecture we divide the network into multiple densely connected dense blocks</p>
  </blockquote>
  
  <p>So, if I understood correctly, the problem is that the feature map size can change, thus we can't concatenate. But how does the addition of transition layers solve this problem?</p>
  
  <p>And how can several dense blocks connected like this be more efficient that one single bigger dense block?</p>
  
  <p>Furthermore, why are all standard DenseNets made of 4 dense blocks? I guess I will have the answer to this question if I understood better the previous questions.</p>
  
Tags: <neural-networks><deep-learning><convolutional-neural-networks><papers><residual-networks>
Title: Why are there transition layers in DenseNet?
UserDisplayName: Astariul
ThreadText: # Why are there transition layers in DenseNet?
  
  (Asked by: username_0 on 2018-10-15)
  
  The [DenseNet architecture](https://arxiv.org/pdf/1608.06993v3.pdf) can be summarizde with this figure:
  [![enter image description here](https://i.sstatic.net/cSwqp.png)](https://i.sstatic.net/cSwqp.png)
  
  **Why there are transition layers between each block?**
  
  In the papers, they justify the use of transition layers as follow :
  
  > The concatenation operation used in Eq. (2) is not viable when the size of feature-maps changes. However, an essential part of convolutional networks is pooling layers that change the size of feature-maps. To facilitate pooling in our architecture we divide the network into multiple densely connected dense blocks
  
  So, if I understood correctly, the problem is that the feature map size can change, thus we can't concatenate. But how does the addition of transition layers solve this problem?
  
  And how can several dense blocks connected like this be more efficient that one single bigger dense block?
  
  Furthermore, why are all standard DenseNets made of 4 dense blocks? I guess I will have the answer to this question if I understood better the previous questions.
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, papers, residual-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-10-15. Score: 1 | Accepted answer
  
  The point of DenseNet was to go as deep as ResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.
  
  About transition layers (convolution + pooling), I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from $56 \times 56$ to $28 \times 28$ to $14 \times 14$, and so on.
  
  The authors state it this way
  
  > To further improve model compactness, we can reduce the number of feature-maps at transition layers
  
  > **Comments:**
  > **username_0**: Thanks. So these transitions layers are here just to reduce complexity ? What is the price of this reduction ? Less precision ?  If we had a really efficient computer, wouldn't it be better to have a single fully connected block ? So every feature map is accessible anywhere in the network ?  
  > **username_1**: Theoretically seems yes, we should do better with fully connected, but we might have to find that out with implementation which doesn't look feasible practically! This is just my conjecture.  
  
  ---
  
  