I’m trying to understand the difference between reduction=‘sum’ and reduction=‘mean’.
From what I understand, with ‘sum’ the loss is summed for every example across every element. For ‘mean’ the loss is summed for every example across every element and then divided by the total amount of examples*elements.
I’m confused why the code below produces different losses:
loss_REC = nn.MSELoss(reduction='sum')
loss_REC_mean = nn.MSELoss(reduction='mean')

L = loss_REC(x_t_hat, x_t)/float(100*64*64) # The shape of x_t is [100, 1, 64, 64]
L_mean = loss_REC_mean(x_t_hat, x_t)

print("L: ", L) 
print("L_mean: ", L_mean)

Results:
L: 0.5480
L_mean: 0.0815