Hi,
All the loss functions in pytorch assume that the first dimension is the batch size.
For few of them like MSELoss, this is not relevant as it just compares every entry in the output and the target one by one and then add them up.
Remember that the loss function formula when you do SGD is: loss = sum_{i} loss(out_i, target_i) where i represent different samples.
The reduce function control whether the sum is done by the loss, meaning you will get a single value as output, or if it is not done, hence you will get a Tensor containing one entry for every element in your batch.