<p>In backpropagation the gradients are used to update the weights using the formula
<span class="math-container">$$w = w - \alpha \frac{dL}{dw}$$</span>
and the loss gradient w.r.t. weights is
<span class="math-container">$$\frac{dL}{dw} = \frac{dL}{dz} \frac{dz}{dw} = (\frac{dL}{da} \frac{da}{dz} \frac{1}{m}) \frac{dz}{dw} $$</span></p>
<p>Why is the there the <span class="math-container">$\frac{1}{m}$</span> term? Does batch size matter and what if it's 1?</p>

CC BY-SA 4.0
2022-07-25T08:19:27.977Z
could you post a link to the book/blogpost from which you got the formula? m should be just the total amount of weights, but the notation is not consistent across literature so to be sure I should check from your source.
Edoardo Guerriero
CC BY-SA 4.0
2022-07-25T22:43:40.007Z
Sorry I forgot to mention that m is the number of features in the last layer's output. Here's the video link https://youtu.be/yXcQ4B-YSjQ?list=PLkDaE6sCZn6Ec-XTbcX1uRg2_u4xOEky0&t=780 but he writes it in an abbreviated way
rkuang25
<neural-networks><tensorflow><backpropagation><math><derivative>
Why Is There The Term 1/m In Backpropagation
rkuang25
# Why Is There The Term 1/m In Backpropagation

(Asked by: username_0 on 2022-07-25)

In backpropagation the gradients are used to update the weights using the formula
$$w = w - \alpha \frac{dL}{dw}$$
and the loss gradient w.r.t. weights is
$$\frac{dL}{dw} = \frac{dL}{dz} \frac{dz}{dw} = (\frac{dL}{da} \frac{da}{dz} \frac{1}{m}) \frac{dz}{dw} $$

Why is the there the $\frac{1}{m}$ term? Does batch size matter and what if it's 1?

**Tags:** neural-networks, tensorflow, backpropagation, math, derivative

> **Comments:**
> **username_1**: could you post a link to the book/blogpost from which you got the formula? m should be just the total amount of weights, but the notation is not consistent across literature so to be sure I should check from your source.  
> **username_0**: Sorry I forgot to mention that m is the number of features in the last layer's output. Here's the video link https://youtu.be/yXcQ4B-YSjQ?list=PLkDaE6sCZn6Ec-XTbcX1uRg2_u4xOEky0&t=780 but he writes it in an abbreviated way  

---

## 0 answers

