Body: <p>At the bottom of page 2 of the paper <a href="https://arxiv.org/abs/1706.05350" rel="nofollow noreferrer">L2 Regularization versus Batch and Weight Normalization</a>, the equation for the gradient of the output with respect to the weights is given as:</p>
  <p><span class="math-container">$$
  \triangledown y_{BN} (X; w, \gamma, \beta)  = \frac{X}{\sigma(X)}\gamma g'(z).
  $$</span></p>
  <p>Can someone break down into smaller steps on how the author got to that equation?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-27T15:50:59.533Z
Comments.Text: Hi and welcome to AI SE! Meanwhile, if you don't receive an answer, what you can do is try to understand the formula step by step. To do that, you first need to understand all the terms in it. For example, what is $X$? It should be the input. $g$ should be the non-linearity? What are all other terms?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-28T05:33:58.187Z
Comments.Text: Watch this: https://youtu.be/d14TUNcbn1k?t=354. If you are still unsure of anything in relation to backprop, respond here and I'd be happy to help
Comments.UserDisplayName: Recessive
Tags: <papers><backpropagation><notation><regularization><batch-normalization>
Title: How is the gradient with respect to weights derived in batch normalization?
UserDisplayName: DeapSoup
ThreadText: # How is the gradient with respect to weights derived in batch normalization?
  
  (Asked by: username_0 on 2020-02-27)
  
  At the bottom of page 2 of the paper [L2 Regularization versus Batch and Weight Normalization](https://arxiv.org/abs/1706.05350), the equation for the gradient of the output with respect to the weights is given as:
  
  $$
  \triangledown y\_{BN} (X; w, \gamma, \beta) = \frac{X}{\sigma(X)}\gamma g'(z).
  $$
  
  Can someone break down into smaller steps on how the author got to that equation?
  
  **Tags:** papers, backpropagation, notation, regularization, batch-normalization
  > **Comments:**
  > **username_1**: Hi and welcome to AI SE! Meanwhile, if you don't receive an answer, what you can do is try to understand the formula step by step. To do that, you first need to understand all the terms in it. For example, what is $X$? It should be the input. $g$ should be the non-linearity? What are all other terms?  
  > **username_2**: Watch this: https://youtu.be/d14TUNcbn1k?t=354. If you are still unsure of anything in relation to backprop, respond here and I'd be happy to help  
  
  ---
  
  ## 0 answers
  
  