Thanks for your reply. I’m creating an instance of the MSELoss class and calling it later (as I mentioned in the original post)



 stark:

criterion = torch.nn.CrossEntropyLoss().cuda()
loss = criterion(torch.nn.functional.softmax(output), target)



but I simply mentioned it as loss = torch.nn.MSELoss(output, target) <-- this staement will cause this error: RuntimeError: bool value of Tensor with more than one value is ambiguous.

I noticed the value returned by MSELoss function was not an average of individual losses of examples in the batch. For instance, considering the batch size = 2, the loss was different from the average of the two losses when batch size was 1 while inputs/ targets were kept the same. Which of these should be given more importance?
a) loss calculated per image (for batch size =1)
b) loss calculated per batch (for batch size > 1)
c) mean loss per epoch (irrespective of the batch size)