Body: <p>Say we have the layer <span class="math-container">$X W + b = Y$</span>.</p>
  
  <ol>
  <li>I want to get <span class="math-container">$\frac{dL}{dW}$</span> and we assume I have <span class="math-container">$\frac{dL}{dY}$</span>.
  So all I need is to find <span class="math-container">$\frac{dY}{dW}$</span>. I know that it should be <span class="math-container">$X^T\frac{dL}{dY}$</span> but don't understand why. please explain.</li>
  <li>I want to get <span class="math-container">$\frac{dL}{db}$</span> and we assume I have <span class="math-container">$\frac{dL}{dY}$</span>.
  So all I need is to find <span class="math-container">$\frac{dY}{db}$</span>. I know that it should be <span class="math-container">$\sum(\frac{dL}{dY})_i$</span> (I mean sum the rows) but I don't understand why. please explain.</li>
  </ol>
  
  <p>Thanks :)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-22T00:55:33.750Z
Comments.Text: The TL;DR answer is that the partial derivative of $L$ with respect to the weights or bias is just an application of the chain rule. Maybe later I'll add a complete answer.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-23T05:15:56.207Z
Comments.Text: Yea I know that it is because of the chain rule. I’m looking for a concise clear proof.
Comments.UserDisplayName: Yuval Kirstain
Tags: <neural-networks><machine-learning><backpropagation><gradient-descent>
Title: Understanding the partial derivative with respect to the weight matrix and bias
UserDisplayName: Yuval Kirstain
ThreadText: # Understanding the partial derivative with respect to the weight matrix and bias
  
  (Asked by: username_0 on 2019-10-19)
  
  Say we have the layer $X W + b = Y$.
  
  1. I want to get $\frac{dL}{dW}$ and we assume I have $\frac{dL}{dY}$.
     So all I need is to find $\frac{dY}{dW}$. I know that it should be $X^T\frac{dL}{dY}$ but don't understand why. please explain.
  2. I want to get $\frac{dL}{db}$ and we assume I have $\frac{dL}{dY}$.
     So all I need is to find $\frac{dY}{db}$. I know that it should be $\sum(\frac{dL}{dY})\_i$ (I mean sum the rows) but I don't understand why. please explain.
  
  Thanks :)
  
  **Tags:** neural-networks, machine-learning, backpropagation, gradient-descent
  > **Comments:**
  > **username_1**: The TL;DR answer is that the partial derivative of $L$ with respect to the weights or bias is just an application of the chain rule. Maybe later I'll add a complete answer.  
  > **username_0**: Yea I know that it is because of the chain rule. I’m looking for a concise clear proof.  
  
  ---
  
  ## 0 answers
  
  