Also no it should not be “sum” because “Loss” documentation specifically says that it expects it to be the average loss. The source code then explains why it says: self._sum += average_loss.item() * n