<p>I'm studying error backpropagation in neural networks. I am interested in why we use only one path on the computational graph to get the value of the derivative for a weight? I ask the question because there are several paths in the computational graph to get the  derivative for a particular weight. Why do we only use a one value? Why don't we combine the values from all possible paths?</p>
<p><strong>Schema</strong>:
<a href="https://i.sstatic.net/1214s.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1214s.png" alt="Backpropagation schema" /></a>
<strong>Fromulas:</strong></p>
<p>Normal path:
<span class="math-container">$$\frac{\partial E}{\partial w_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a_{1,1}}\cdot  \frac{\partial a_{1,1}}{\partial a_{0,1}}\cdot  \frac{\partial a_{0,1}}{\partial w_{1,1}}$$</span></p>
<p>Alternate path:
Normal path:
<span class="math-container">$$\frac{\partial E}{\partial w_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a_{1,2}}\cdot  \frac{\partial a_{1,2}}{\partial a_{0,1}}\cdot  \frac{\partial a_{0,1}}{\partial w_{1,1}}$$</span></p>
<p>Why don't we consider both derivatives or the sum of them?</p>

CC BY-SA 4.0
2021-09-06T10:50:10.693Z
All of them are calculated and added together.
Criticize SE actions means ban
<machine-learning><deep-learning><backpropagation><gradient-descent>
Different ways to calculate backpropagation derivatives, any difference?
Mike Like
# Different ways to calculate backpropagation derivatives, any difference?

(Asked by: username_0 on 2021-09-05)

I'm studying error backpropagation in neural networks. I am interested in why we use only one path on the computational graph to get the value of the derivative for a weight? I ask the question because there are several paths in the computational graph to get the derivative for a particular weight. Why do we only use a one value? Why don't we combine the values from all possible paths?

**Schema**:
[![Backpropagation schema](https://i.sstatic.net/1214s.png)](https://i.sstatic.net/1214s.png)
**Fromulas:**

Normal path:
$$\frac{\partial E}{\partial w\_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a\_{1,1}}\cdot \frac{\partial a\_{1,1}}{\partial a\_{0,1}}\cdot \frac{\partial a\_{0,1}}{\partial w\_{1,1}}$$

Alternate path:
Normal path:
$$\frac{\partial E}{\partial w\_{1,1}} = \frac{\partial E}{\partial Out} \cdot \frac{\partial Out}{\partial a\_{1,2}}\cdot \frac{\partial a\_{1,2}}{\partial a\_{0,1}}\cdot \frac{\partial a\_{0,1}}{\partial w\_{1,1}}$$

Why don't we consider both derivatives or the sum of them?

**Tags:** machine-learning, deep-learning, backpropagation, gradient-descent

> **Comments:**
> **username_1**: All of them are calculated and added together.  

---

## 0 answers

