Body: <p>Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer <code>l</code>, in a neural network to particular nodes in the previous layer <code>l-1</code> of the neural network as well as to particular nodes in the next layer <code>l+1</code> of the neural network?</p>
  
  <p>This is obviously contrived, but here is a illustration of what I mean.
  <a href="https://i.sstatic.net/Vy7FL.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Vy7FL.jpg" alt="enter image description here"></a>
  The thick line with arrow indecates an edge that leads to l-1 layer neuron</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-01T08:11:47.210Z
Comments.Text: Yes. Feedback ANN and Recurrent ANN are types of what you want.
Comments.UserDisplayName: OmG
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-01T13:15:15.323Z
Comments.Text: Maybe make your question clearer with a diagram? I was not imagining RNNs as an answer, but ResNets with skip connections. Both RNNs and skip connections are useful, in very different ways, and could be explained in an answer.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-02T10:54:07.167Z
Comments.Text: Thanks for adding the diagram. This is closer to RNN in design as it involves loops, and thus necessarily some kind of time step parameter in order to decide which value to feed. I'm not sure how much that kind of feedback connection has been studied, and what name(s) it goes under though.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-02T18:53:59.637Z
Comments.Text: @OmG It looks like recurrent ANNs are the answer the question author was looking for, thanks! Would you like to expand your comment into an answer with some background/explanation?
Comments.UserDisplayName: Ben N
Tags: <neural-networks><machine-learning><architecture>
Title: Algorithms that connect neurons to previous layers as well as next
UserDisplayName: benbyford
ThreadText: # Algorithms that connect neurons to previous layers as well as next
  
  (Asked by: username_0 on 2017-12-31)
  
  Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer `l`, in a neural network to particular nodes in the previous layer `l-1` of the neural network as well as to particular nodes in the next layer `l+1` of the neural network?
  
  This is obviously contrived, but here is a illustration of what I mean.
  [![enter image description here](https://i.sstatic.net/Vy7FL.jpg)](https://i.sstatic.net/Vy7FL.jpg)
  The thick line with arrow indecates an edge that leads to l-1 layer neuron
  
  **Tags:** neural-networks, machine-learning, architecture
  > **Comments:**
  > **username_1**: Yes. Feedback ANN and Recurrent ANN are types of what you want.  
  > **username_2**: Maybe make your question clearer with a diagram? I was not imagining RNNs as an answer, but ResNets with skip connections. Both RNNs and skip connections are useful, in very different ways, and could be explained in an answer.  
  > **username_2**: Thanks for adding the diagram. This is closer to RNN in design as it involves loops, and thus necessarily some kind of time step parameter in order to decide which value to feed. I'm not sure how much that kind of feedback connection has been studied, and what name(s) it goes under though.  
  > **username_3**: @OmG It looks like recurrent ANNs are the answer the question author was looking for, thanks! Would you like to expand your comment into an answer with some background/explanation?  
  
  ---
  
  ## 0 answers
  
  