Hi,
Pytorch computes the gradients wrt the loss you computed. So if your loss is the average, then the gradients will be the average as well.
If you don’t average the loss, then you will get as many losses as the batch size. And you will need to run as many backwards as the batch size to get all these gradients.