Body: <p>I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm.</p>
  <p>In back propagation, I understand we want to go backwards from the last neurons to adjust the weights and biases to that predicted final neurons. For it to calculate error and derivative, It needs to have the last inputs, the predicted output based on the layer's weights and the actual value ( target ).</p>
  <p>As In the final neuron layers we have all this information. But how do we calculate the inputs of middle and hidden layers?</p>
  <p><a href="https://i.sstatic.net/VHwrz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/VHwrz.png" alt="Simple Neural Network with 2 hidden layers" /></a></p>
  <p>Suppose we have the final output ( 0.73 ), we calculate the error and derivatives of W31, W32 and W33; and adjust them to match the final output, Then we shall go one layer back in our network.</p>
  <p>Now we need the N11, N12, N13 and N14 values and the target values of N21, N22 and N23 to calculate errors and derivatives, but we don't have them</p>
  <p>Should we feed forward the whole network and map all the labels and values of each neuron in memory to be able to access it later? Because it would be very, very memory and resource intensive on large networks.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-27T18:32:38.633Z
Comments.Text: You can take a look at this site: [backpropagation process](http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html) There is a whole process of backpropagation with examples.
Comments.UserDisplayName: ofevy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-28T12:09:38.720Z
Comments.Text: These two videos give a good intuition about how backpropagation works: https://www.youtube.com/watch?v=Ilg3gGewQ5U https://www.youtube.com/watch?v=yXcQ4B-YSjQ The second video also focuses on how it would be implemented in code.
Comments.UserDisplayName: Harry Min
Tags: <neural-networks><deep-learning><backpropagation><deep-neural-networks><hidden-layers>
Title: How does back propagation adjust the hidden layers' weights and biases?
UserDisplayName: KiarashRzg
ThreadText: # How does back propagation adjust the hidden layers' weights and biases?
  
  (Asked by: username_0 on 2021-08-27)
  
  I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm.
  
  In back propagation, I understand we want to go backwards from the last neurons to adjust the weights and biases to that predicted final neurons. For it to calculate error and derivative, It needs to have the last inputs, the predicted output based on the layer's weights and the actual value ( target ).
  
  As In the final neuron layers we have all this information. But how do we calculate the inputs of middle and hidden layers?
  
  [![Simple Neural Network with 2 hidden layers](https://i.sstatic.net/VHwrz.png)](https://i.sstatic.net/VHwrz.png)
  
  Suppose we have the final output ( 0.73 ), we calculate the error and derivatives of W31, W32 and W33; and adjust them to match the final output, Then we shall go one layer back in our network.
  
  Now we need the N11, N12, N13 and N14 values and the target values of N21, N22 and N23 to calculate errors and derivatives, but we don't have them
  
  Should we feed forward the whole network and map all the labels and values of each neuron in memory to be able to access it later? Because it would be very, very memory and resource intensive on large networks.
  
  **Tags:** neural-networks, deep-learning, backpropagation, deep-neural-networks, hidden-layers
  
  > **Comments:**
  > **username_1**: You can take a look at this site: [backpropagation process](http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html) There is a whole process of backpropagation with examples.  
  > **username_2**: These two videos give a good intuition about how backpropagation works: https://www.youtube.com/watch?v=Ilg3gGewQ5U https://www.youtube.com/watch?v=yXcQ4B-YSjQ The second video also focuses on how it would be implemented in code.  
  
  ---
  
  ## 0 answers
  
  