Hello Dhorka!



 Dhorka:

At this moment the documentation says:

‘mean’: the sum of the output will be divided by the number of elements in the output,



Yes, you’re right about this.  Read literally, the current documentation
for CrossEntropyLoss is incorrect.
The line you cite is wrong (or, at best, imprecise or incomplete).
Also, when read strictly, this passage:

or in the case of the weight argument being specified:
{loss}(x, class) = weight[class] (…)
The losses are averaged across observations for each minibatch.

is incorrect because it doesn’t show the final reduced loss being
divided by the sum of the weights.
I believe I’ve seen this issue – whether the average for weighted
loss is divided by the sum of the weights – come up in this forum
before.  (I don’t have the references off hand.)
@ptrblck, would it make sense to have the Documentation Mavens
clarify this?   (Also, it looks like BCELoss 1 and BCEWithLogitsLoss
get this wrong, while NLLLoss gets it right.)
Best.
K. Frank