<p>Lateral connection in NN simply means that units in a hidden layer are connected with one another. Suppose we have a hidden layer Hi which has 10 from (1,2,....10) then lateral connection implies that unit 1 may be connected to unit 2.</p>
<p>So now the activation of unit 2 is not only dependent on incoming inputs but also on unit 1. The weight assigned to a connection between unit 2 and unit 1 needs to be learned as part of backprop.</p>
<p>Please refer to the below image for more details :</p>
<p><a href="https://i.sstatic.net/U8K0H.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/U8K0H.png" alt="enter image description here" /></a></p>
<blockquote>
<p>Lateral connections in a feed-forward architecture. Inputs and hidden layer neurons are fully connected as are hidden layer and output layer neurons. Neuron j in the hidden layer also receives the net input of neuron (j - 1) in the hidden layer through a lateral connection</p>
</blockquote>
<p>Ref: <a href="https://www.semanticscholar.org/paper/On-lateral-connections-in-feed-forward-neural-Kothari-Agyepong/167d7c20dcf37fba0388b0365d9bd42ba8b2ecde/figure/0" rel="nofollow noreferrer">https://www.semanticscholar.org/paper/On-lateral-connections-in-feed-forward-neural-Kothari-Agyepong/167d7c20dcf37fba0388b0365d9bd42ba8b2ecde/figure/0</a></p>
<p><strong>Lateral Connection can occur between units in layers and can be applied to feed forward network</strong></p>

CC BY-SA 4.0
2022-01-06T12:57:39.997Z
2022-05-22T11:38:46.823Z
2022-05-22T11:38:46.823Z
Ashwiniku918
<p>A class of CNN is popular due to the implementation of residual connections.</p>
<p>We can use both terms &quot;residual connections&quot; and &quot;skip connections&quot; interchangeably <a href="https://stats.stackexchange.com/a/321112/42883">as they refer to the same.</a></p>
<blockquote>
<p>Residual connections are the same thing as 'skip connections'. They
are used to allow gradients to flow through a network directly,
without passing through non-linear activation functions. Non-linear
activation functions, by nature of being non-linear, cause the
gradients to explode or vanish (depending on the weights).</p>
</blockquote>
<p>While studying some models that are used in codes of some deep learning projects, I came across the word &quot;lateral connections&quot;.</p>
<p>I guess that lateral connections stand for the connections that are not present in the traditional feed-forward neural networks of any kind. Lateral connections may be from any layer to any other layer.</p>
<p>Am I true? If not, when can I call a connection lateral?</p>

CC BY-SA 4.0
2022-01-06T13:00:13.113Z
When you say "While studying some models that are used in codes of some deep learning projects", it may be a good idea to provide the source where you read about this term.
nbro
<neural-networks><terminology>
What is meant by "lateral connection" in the context of neural networks?
hanugm
# What is meant by "lateral connection" in the context of neural networks?

(Asked by: username_0 on 2022-01-06)

A class of CNN is popular due to the implementation of residual connections.

We can use both terms "residual connections" and "skip connections" interchangeably [as they refer to the same.](https://stats.stackexchange.com/a/321112/42883)

> Residual connections are the same thing as 'skip connections'. They
> are used to allow gradients to flow through a network directly,
> without passing through non-linear activation functions. Non-linear
> activation functions, by nature of being non-linear, cause the
> gradients to explode or vanish (depending on the weights).

While studying some models that are used in codes of some deep learning projects, I came across the word "lateral connections".

I guess that lateral connections stand for the connections that are not present in the traditional feed-forward neural networks of any kind. Lateral connections may be from any layer to any other layer.

Am I true? If not, when can I call a connection lateral?

**Tags:** neural-networks, terminology

> **Comments:**
> **username_1**: When you say "While studying some models that are used in codes of some deep learning projects", it may be a good idea to provide the source where you read about this term.  

---

## 1 answers

### Answer by username_2 on 2022-01-06. Score: 3 | Accepted answer

Lateral connection in NN simply means that units in a hidden layer are connected with one another. Suppose we have a hidden layer Hi which has 10 from (1,2,....10) then lateral connection implies that unit 1 may be connected to unit 2.

So now the activation of unit 2 is not only dependent on incoming inputs but also on unit 1. The weight assigned to a connection between unit 2 and unit 1 needs to be learned as part of backprop.

Please refer to the below image for more details :

[![enter image description here](https://i.sstatic.net/U8K0H.png)](https://i.sstatic.net/U8K0H.png)

> Lateral connections in a feed-forward architecture. Inputs and hidden layer neurons are fully connected as are hidden layer and output layer neurons. Neuron j in the hidden layer also receives the net input of neuron (j - 1) in the hidden layer through a lateral connection

Ref: <https://www.semanticscholar.org/paper/On-lateral-connections-in-feed-forward-neural-Kothari-Agyepong/167d7c20dcf37fba0388b0365d9bd42ba8b2ecde/figure/0>

**Lateral Connection can occur between units in layers and can be applied to feed forward network**


---

