<p>What you stated looks correct :-</p>
<p>The SSE loss function includes the summation of the errors in the other training examples of the mini-batch (each training example's loss in the mini-batch is summed for one big loss)
SSE = Summation of (y-y')squared for all samples in mini-batch</p>
<p>The MSE loss function averages the loss of each individual training example in the mini-batch, and then all those losses are averaged based on the mini-batch size.
MSE = (1/m)SSE</p>

CC BY-SA 4.0
2022-01-31T12:36:21.367Z
You can format the formulas and math symbols in this answer with latex. I would recommend that you do that.
nbro
CC BY-SA 4.0
2022-01-29T16:57:36.027Z
2022-01-29T16:57:36.027Z
Vivek Singhal
<p>I've been trying to implement my own neural network library and have been wondering if:</p>
<ul>
<li><p>The SSE loss function includes the summation of the errors in the other training examples of the mini-batch (each training example's loss in the mini-batch is summed for one big loss)</p>
</li>
<li><p>The MSE loss function averages the loss of each individual training example in the mini-batch, and then all those losses are averaged based on the mini-batch size.</p>
</li>
</ul>
<p>or if the summing or averaging of the weight gradients has nothing to do with the loss function used?</p>
<p>I feel like the answer would be clear if I knew other loss functions, if it did matter that would mean the weight gradients should be averaged for MSE and summed for SSE in mini-batch gradient descent?</p>

CC BY-SA 4.0
2022-01-28T15:51:32.967Z
seems at least a partial dup of [this](https://ai.stackexchange.com/q/20377/2444). let me know.
nbro
CC BY-SA 4.0
2022-01-28T16:03:31.710Z
hi, I've seen that one before. I don't think it answers my question.
jake_prentice
<neural-networks><objective-functions><mini-batch-gradient-descent>
Does the summing or averaging of the weight gradients have anything to do with the cost function used?
jake_prentice
# Does the summing or averaging of the weight gradients have anything to do with the cost function used?

(Asked by: username_0 on 2022-01-28)

I've been trying to implement my own neural network library and have been wondering if:

* The SSE loss function includes the summation of the errors in the other training examples of the mini-batch (each training example's loss in the mini-batch is summed for one big loss)
* The MSE loss function averages the loss of each individual training example in the mini-batch, and then all those losses are averaged based on the mini-batch size.

or if the summing or averaging of the weight gradients has nothing to do with the loss function used?

I feel like the answer would be clear if I knew other loss functions, if it did matter that would mean the weight gradients should be averaged for MSE and summed for SSE in mini-batch gradient descent?

**Tags:** neural-networks, objective-functions, mini-batch-gradient-descent

> **Comments:**
> **username_1**: seems at least a partial dup of [this](https://ai.stackexchange.com/q/20377/2444). let me know.  
> **username_0**: hi, I've seen that one before. I don't think it answers my question.  

---

## 1 answers

### Answer by username_2 on 2022-01-29. Score: 1 | Accepted answer

What you stated looks correct :-

The SSE loss function includes the summation of the errors in the other training examples of the mini-batch (each training example's loss in the mini-batch is summed for one big loss)
SSE = Summation of (y-y')squared for all samples in mini-batch

The MSE loss function averages the loss of each individual training example in the mini-batch, and then all those losses are averaged based on the mini-batch size.
MSE = (1/m)SSE

> **Comments:**
> **username_1**: You can format the formulas and math symbols in this answer with latex. I would recommend that you do that.  

---

