So while working on a Language model I was testing some individual values with nn.CrossEntropyLoss
toyloss = nn.CrossEntropyLoss(reduction='none')

toyinput = torch.zeros(1, 5)
toytarget = torch.zeros(1, dtype=torch.long)

toytarget[0] = 2
toyinput[0,2] = 1
toyoutput = toyloss(toyinput, toytarget)

print(toyoutput)

In short the input and targets are:

tensor([[0., 0., 1., 0., 0.]], tensor([2])

respectively, and the loss is:

tensor([0.9048])

My question is why is the loss 0.9048 if the prediction completely matches the target?
My guess is that it has something to do with the fact that this Loss implements logSoftmax, but then, what should we the correct interpretation for this result?
Is 0.9048 the minimum loss possible when using this criterion?
Edit: I decided to try using only NLLLoss and I’ve got a loss of

tensor([-1.])

Honestly, I was expecting it to be 0… Am I missing something about the definition of Loss? I mean aren’t we looking for the closest value to 0?