Let’s assume, there are 10 examples in a dataset and the batch size is 4.
The number of exmaples per batch and training loss will be:

Note: By default nn.MSELoss() returns mean loss for an entire batch!
Which of these is the loss for an epoch?
A.  Mean of individual losses over the training dataset:
Σ(mean_batch_loss * batch_size) / Σ(batch_size)
= ((27.527*4)+(10.503*4)+(5.6534*2)) / (4+4+2)
= (110.11 + 42.0116 + 11.3068) / 10
= 163.428 / 10 = 16.3428

B. Mean of the mean batch losses:
Σ mean_batch_loss / total_batches
= (27.527 + 10.503 + 5.6534*2) / total_batches
= 43.6837 / 3 
= 14.5612

C. Exponentially weighted moving average (EWMA)
s(i) = a * x(i) + (1-a) * x(i-1)
where a is a smoothing factor set to 0.1 and s(0) = 27.527
s(0) = 27.527
s(1) = 25.825
s(2) = 23.808
D. Differently than above
Please also comment on the way validation loss is computed differently.