According to nll_loss documentation, for reduction parameter, " 'none' : no reduction will be applied,  'mean' : the sum of the output will be divided by the number of elements in the output,  'sum' : the output will be summed."
However, it seems “mean” is divided by the sum of the weights of each element, not number of elements in the output. Am I misunderstanding the documentation or the documentation is inaccurate?



input = torch.randn(3, 5, requires_grad=True)
ls =F.log_softmax(input, 1)
ls
tensor([[-2.5076, -2.2387, -0.4952, -2.2435, -2.3390],
[-0.9897, -1.8469, -1.6642, -1.4942, -2.8676],
[-0.7173, -3.8274, -2.2838, -1.0784, -3.0335]],
grad_fn=)




F.nll_loss(ls, torch.tensor([0,0,4]), reduction=“none”, weight=torch.tensor([1.0, 1.0, 1.0, 1.0, 3.0]))



tensor([2.5076, 0.9897, 9.1005], grad_fn=)



F.nll_loss(ls, torch.tensor([0,0,4]), reduction=“sum”, weight=torch.tensor([1.0, 1.0, 1.0, 1.0, 3.0]))



tensor(12.5978, grad_fn=)



F.nll_loss(ls, torch.tensor([0,0,4]), reduction=“mean”, weight=torch.tensor([1.0, 1.0, 1.0, 1.0, 3.0]))



tensor(2.5196, grad_fn=)