<p>Say we have the layer <span class="math-container">$X W + b = Y$</span>.</p>

<ol>
<li>I want to get <span class="math-container">$\frac{dL}{dW}$</span> and we assume I have <span class="math-container">$\frac{dL}{dY}$</span>.
So all I need is to find <span class="math-container">$\frac{dY}{dW}$</span>. I know that it should be <span class="math-container">$X^T\frac{dL}{dY}$</span> but don't understand why. please explain.</li>
<li>I want to get <span class="math-container">$\frac{dL}{db}$</span> and we assume I have <span class="math-container">$\frac{dL}{dY}$</span>.
So all I need is to find <span class="math-container">$\frac{dY}{db}$</span>. I know that it should be <span class="math-container">$\sum(\frac{dL}{dY})_i$</span> (I mean sum the rows) but I don't understand why. please explain.</li>
</ol>

<p>Thanks :)</p>

CC BY-SA 4.0
2019-10-22T00:55:33.750Z
The TL;DR answer is that the partial derivative of $L$ with respect to the weights or bias is just an application of the chain rule. Maybe later I'll add a complete answer.
nbro
CC BY-SA 4.0
2019-10-23T05:15:56.207Z
Yea I know that it is because of the chain rule. I’m looking for a concise clear proof.
Yuval Kirstain
<neural-networks><machine-learning><backpropagation><gradient-descent>
Understanding the partial derivative with respect to the weight matrix and bias
Yuval Kirstain
# Understanding the partial derivative with respect to the weight matrix and bias

(Asked by: username_0 on 2019-10-19)

Say we have the layer $X W + b = Y$.

1. I want to get $\frac{dL}{dW}$ and we assume I have $\frac{dL}{dY}$.
   So all I need is to find $\frac{dY}{dW}$. I know that it should be $X^T\frac{dL}{dY}$ but don't understand why. please explain.
2. I want to get $\frac{dL}{db}$ and we assume I have $\frac{dL}{dY}$.
   So all I need is to find $\frac{dY}{db}$. I know that it should be $\sum(\frac{dL}{dY})\_i$ (I mean sum the rows) but I don't understand why. please explain.

Thanks :)

**Tags:** neural-networks, machine-learning, backpropagation, gradient-descent

> **Comments:**
> **username_1**: The TL;DR answer is that the partial derivative of $L$ with respect to the weights or bias is just an application of the chain rule. Maybe later I'll add a complete answer.  
> **username_0**: Yea I know that it is because of the chain rule. I’m looking for a concise clear proof.  

---

## 0 answers

