Body: <p>I'm studying error backpropagation in neural networks. I am interested in why we use only one path on the computational graph to get the value of the derivative for a weight? I ask the question because there are several paths in the computational graph to get the  derivative for a particular weight. Why do we only use a one value? Why don't we combine the values from all possible paths?</p>
  <p><strong>Schema</strong>:
  <a href="https://i.sstatic.net/1214s.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1214s.png" alt="Backpropagation schema" /></a>
  <strong>Fromulas:</strong></p>
  <p>Normal path:
  <span class="math-container">$$\frac{\partial E}{\partial w_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a_{1,1}}\cdot  \frac{\partial a_{1,1}}{\partial a_{0,1}}\cdot  \frac{\partial a_{0,1}}{\partial w_{1,1}}$$</span></p>
  <p>Alternate path:
  Normal path:
  <span class="math-container">$$\frac{\partial E}{\partial w_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a_{1,2}}\cdot  \frac{\partial a_{1,2}}{\partial a_{0,1}}\cdot  \frac{\partial a_{0,1}}{\partial w_{1,1}}$$</span></p>
  <p>Why don't we consider both derivatives or the sum of them?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-06T10:50:10.693Z
Comments.Text: All of them are calculated and added together.
Comments.UserDisplayName: Criticize SE actions means ban
Tags: <machine-learning><deep-learning><backpropagation><gradient-descent>
Title: Different ways to calculate backpropagation derivatives, any difference?
UserDisplayName: Mike Like
ThreadText: # Different ways to calculate backpropagation derivatives, any difference?
  
  (Asked by: username_0 on 2021-09-05)
  
  I'm studying error backpropagation in neural networks. I am interested in why we use only one path on the computational graph to get the value of the derivative for a weight? I ask the question because there are several paths in the computational graph to get the derivative for a particular weight. Why do we only use a one value? Why don't we combine the values from all possible paths?
  
  **Schema**:
  [![Backpropagation schema](https://i.sstatic.net/1214s.png)](https://i.sstatic.net/1214s.png)
  **Fromulas:**
  
  Normal path:
  $$\frac{\partial E}{\partial w\_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a\_{1,1}}\cdot \frac{\partial a\_{1,1}}{\partial a\_{0,1}}\cdot \frac{\partial a\_{0,1}}{\partial w\_{1,1}}$$
  
  Alternate path:
  Normal path:
  $$\frac{\partial E}{\partial w\_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a\_{1,2}}\cdot \frac{\partial a\_{1,2}}{\partial a\_{0,1}}\cdot \frac{\partial a\_{0,1}}{\partial w\_{1,1}}$$
  
  Why don't we consider both derivatives or the sum of them?
  
  **Tags:** machine-learning, deep-learning, backpropagation, gradient-descent
  > **Comments:**
  > **username_1**: All of them are calculated and added together.  
  
  ---
  
  ## 0 answers
  
  