Yes, that’s true. The important thing is that calling loss.backward() only works on scalar values. Using the reduction=mean or reduction=sum can take care of this. But sometimes, one may need to use reduction=none in order to apply different mean/sum along different dimensions, for example, applying mean over the batch but taking the sum over pixels.