Back Propagation
[Gradient Descent](https://texonom.com/gradient-descent-c1342b13182f4fb6959023a75b5e2ff8) [Parameter Estimation](https://texonom.com/parameter-estimation-2c5667f2707b4d3082e96a5928464029) [Neuroplasticity](https://texonom.com/neuroplasticity-63de0a67105e4c6ab8f6958b281c4e19) 


### Calculate the partial Derivatives via chain-rule
$$\frac{\partial L}{\partial W_L} = \frac{\partial L}{\partial h_L} \frac{\partial h_L}{\partial W_L}$$
Compute the gradient of [Loss Function](https://texonom.com/loss-function-e8f6343914494828988137987cf459f9) and update weights [Chain Rule](https://texonom.com/chain-rule-cdc4287f5b2a4bcda97886907836d5d0) 
[Computational Graph](https://texonom.com/computational-graph-51f54bce30ed44569be80f56d596340f), H is hidden layer
$$downstream \; gradient = upstream \; gradient \times  local \; gradient$$
Not a same path of [Computational Graph](https://texonom.com/computational-graph-51f54bce30ed44569be80f56d596340f), no reuse, function part also has one derivative chain
> ![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fb34eb922-2e01-47e5-8776-f7a0fdbad46f%2FUntitled.png?table=block&id=e2159703-2d35-487f-be11-e509de807104&cache=v2)

> ![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F71cadd9c-986d-4ac9-b197-1f4603a77806%2FUntitled.png?table=block&id=b56b012e-fde2-4034-b9a1-be7c356345d7&cache=v2)

### Back Propagation Notion
|Title|
|:-:|
|[Forward Propagation](https://texonom.com/forward-propagation-59704e8a904b464fabc23519ae4875f2)|
|[Vanishing Gradient](https://texonom.com/vanishing-gradient-6dc21442840841b3879d6502c5efd98a)|
|[Back Propagation History](https://texonom.com/back-propagation-history-caed7f53c8b546e394b18f81390f924d)|







