Answers.Body: <p>I think this is one of those vague terminologies used in the context of skip (long-range connections). In a standard feedforward network, say if the information needs to propagate from layer 1 to 4, it has to go through two intermediate layers, namely <span class="math-container">$1 \to 2\to 3\to 4$</span>. In a densely connected convolution network, every layer is connected to every layer downstream, meaning that you have a <strong>direct</strong> connection from layer 1 to 4, or <span class="math-container">$1\to 4$</span>, hence a &quot;shorter connection&quot;.</p>
  <p>Note you also have other &quot;shorter connections&quot; such as <span class="math-container">$1\to 3\to 4$</span> and <span class="math-container">$1 \to 2 \to 4$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-19T02:57:21.680Z
Answers.LastActivityDate: 2022-10-19T02:57:21.680Z
Answers.UserDisplayName: PeaBrane
Answers.Body: <p>Shorter connections are nothing but concatenating output tensors along the channel axis. Having shorter connections between layers, will help in gradient flow throughout the network easily and hence better training.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-23T04:27:09.187Z
Answers.LastActivityDate: 2021-09-23T04:27:09.187Z
Answers.UserDisplayName: spb
Body: <p>Consider the following two excerpts from the research paper titled <a href="https://openaccess.thecvf.com/content_cvpr_2017/papers/Huang_Densely_Connected_Convolutional_CVPR_2017_paper.pdf" rel="nofollow noreferrer">Densely Connected Convolutional Networks</a> by <em>Gao Huang et al</em>.</p>
  <p>#1: From abstract</p>
  <blockquote>
  <p>Recent work has shown that convolutional networks can be substantially
  deeper, more accurate, and efficient to train if they contain <strong>shorter
  connections</strong> between layers close to the input and those close to the
  output.</p>
  </blockquote>
  <p>#2: From discussion</p>
  <blockquote>
  <p>One explanation for the improved accuracy of dense convolutional
  networks may be that individual layers receive additional supervision
  from the loss function through the <strong>shorter connections</strong>. One can
  interpret DenseNets to perform a kind of “deep supervision”.</p>
  </blockquote>
  <p>Both excerpts mention the type of connections called shorter connections, especially to the layers that are close to the input and the output layers of the deep convolutional neural network. What does it mean by <strong>shorter connections</strong> here?</p>
  
Tags: <convolutional-neural-networks>
Title: What is meant by "shorter connections" in the case of deep convolutional neural networks?
UserDisplayName: hanugm
ThreadText: # What is meant by "shorter connections" in the case of deep convolutional neural networks?
  
  (Asked by: username_0 on 2021-09-23)
  
  Consider the following two excerpts from the research paper titled [Densely Connected Convolutional Networks](https://openaccess.thecvf.com/content_cvpr_2017/papers/Huang_Densely_Connected_Convolutional_CVPR_2017_paper.pdf) by *Gao Huang et al*.
  
  #1: From abstract
  
  > Recent work has shown that convolutional networks can be substantially
  > deeper, more accurate, and efficient to train if they contain **shorter
  > connections** between layers close to the input and those close to the
  > output.
  
  #2: From discussion
  
  > One explanation for the improved accuracy of dense convolutional
  > networks may be that individual layers receive additional supervision
  > from the loss function through the **shorter connections**. One can
  > interpret DenseNets to perform a kind of “deep supervision”.
  
  Both excerpts mention the type of connections called shorter connections, especially to the layers that are close to the input and the output layers of the deep convolutional neural network. What does it mean by **shorter connections** here?
  
  **Tags:** convolutional-neural-networks
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-10-19. Score: 1
  
  I think this is one of those vague terminologies used in the context of skip (long-range connections). In a standard feedforward network, say if the information needs to propagate from layer 1 to 4, it has to go through two intermediate layers, namely $1 \to 2\to 3\to 4$. In a densely connected convolution network, every layer is connected to every layer downstream, meaning that you have a **direct** connection from layer 1 to 4, or $1\to 4$, hence a "shorter connection".
  
  Note you also have other "shorter connections" such as $1\to 3\to 4$ and $1 \to 2 \to 4$
  
  
  ---
  
  ### Answer by username_2 on 2021-09-23. Score: 0
  
  Shorter connections are nothing but concatenating output tensors along the channel axis. Having shorter connections between layers, will help in gradient flow throughout the network easily and hence better training.
  
  
  ---
  
  