Answers.Body: <p>The neural Network in the image is a &quot;<a href="https://en.wikipedia.org/wiki/Recurrent_neural_network" rel="nofollow noreferrer">Recurrent Neural Network</a>&quot;(RNN). Because of the connection leading backward from <code>h10</code> to <code>h01</code>, <code>h10</code> has to be a &quot;memory node&quot; (mn), meaning it can store its value from the previous input. The basic functionality of an RNN can be seen in this animation:</p>
  <hr />
  <p><img src="https://i.sstatic.net/v0P9X.gif" alt="animation" /></p>
  <hr />
  <h3>Your example:</h3>
  <p>In the beginning, the storage of the mn is initialized with a value, probably 0.<br />
  Now the first input is fed into network:</p>
  <ul>
  <li><code>i0</code> = 2</li>
  <li><code>i1</code> = 3</li>
  <li><code>h00</code> = (<code>i0</code> * 0.4) = 0.8</li>
  <li><code>h01</code> = (<code>i1</code> * -0.9) + (&quot;the stored value of <code>h10</code>&quot; * 1.2) = -2.7<br />
  (&quot;the stored value of <code>h10</code>&quot; in the first run is 0.)</li>
  <li><code>h10</code> = (<code>h00</code> * 0.85) + (<code>h01</code> * -0.2) = 1.22</li>
  <li><code>out</code> = (<code>h10</code> * 0.3) + (<code>h01</code> * 0.1) =  0.096</li>
  </ul>
  <p>Now you can feed the next input through the network and use -2.7 as &quot;the stored value of <code>h10</code>&quot; and so on. You can also add an activation function as you would for any other NN.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-31T13:23:48.097Z
Answers.Text: Why is `out` calculated later than `h10`? Technically, `out` and `h10` is on the same layer, just like `h00` is to `h01`. What is the algorithm for the calculation order?
Answers.UserDisplayName: andras
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-03-30T19:18:07.203Z
Answers.LastActivityDate: 2023-04-22T18:15:02.153Z
Answers.LastEditDate: 2023-04-22T18:15:02.153Z
Answers.UserDisplayName: Nigk
Body: <p>What is the output value of the network for these inputs respectively, and why?
  (Linear activation function is fine.)</p>
  
  <p>[2, 3][-1, 2][1, 0][3, 4]</p>
  
  <p>My main question is how you take the 'backwards' directed paths into account.</p>
  
  <p><a href="https://i.sstatic.net/H1xj8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/H1xj8.png" alt="enter image description here"></a></p>
  
Tags: <neural-networks><recurrent-neural-networks>
Title: How to calculate the output of this neural network?
UserDisplayName: andras
ThreadText: # How to calculate the output of this neural network?
  
  (Asked by: username_0 on 2018-03-29)
  
  What is the output value of the network for these inputs respectively, and why?
  (Linear activation function is fine.)
  
  [2, 3][-1, 2][1, 0][3, 4]
  
  My main question is how you take the 'backwards' directed paths into account.
  
  [![enter image description here](https://i.sstatic.net/H1xj8.png)](https://i.sstatic.net/H1xj8.png)
  
  **Tags:** neural-networks, recurrent-neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-03-30. Score: 1
  
  The neural Network in the image is a "[Recurrent Neural Network](https://en.wikipedia.org/wiki/Recurrent_neural_network)"(RNN). Because of the connection leading backward from `h10` to `h01`, `h10` has to be a "memory node" (mn), meaning it can store its value from the previous input. The basic functionality of an RNN can be seen in this animation:
  
  ---
  
  ![animation](https://i.sstatic.net/v0P9X.gif)
  
  ---
  
  ### Your example:
  
  In the beginning, the storage of the mn is initialized with a value, probably 0.  
  Now the first input is fed into network:
  
  * `i0` = 2
  * `i1` = 3
  * `h00` = (`i0` \* 0.4) = 0.8
  * `h01` = (`i1` \* -0.9) + ("the stored value of `h10`" \* 1.2) = -2.7  
    ("the stored value of `h10`" in the first run is 0.)
  * `h10` = (`h00` \* 0.85) + (`h01` \* -0.2) = 1.22
  * `out` = (`h10` \* 0.3) + (`h01` \* 0.1) = 0.096
  
  Now you can feed the next input through the network and use -2.7 as "the stored value of `h10`" and so on. You can also add an activation function as you would for any other NN.
  
  > **Comments:**
  > **username_0**: Why is `out` calculated later than `h10`? Technically, `out` and `h10` is on the same layer, just like `h00` is to `h01`. What is the algorithm for the calculation order?  
  
  ---
  
  