Hello Matheus!



 mfcs:

I was performing some tests here and result of the Cross Entropy Loss in PyTorch doesn’t match with the result using the expression below:



The issue is that pytorch’s CrossEntropyLoss doesn’t exactly match
the conventional definition of cross-entropy that you gave above.
Rather, it expects raw-score logits as it inputs, and, in effect, applies
softmax() to the logits internally to convert them to probabilities.
(CrossEntropyLoss might better have been named
CrossEntropyWithLogitsLoss.)


I am trying this example here using Cross Entropy Loss from PyTorch:
probs1 = torch.tensor([[
    [
      [ 0.1, 0.3],
      [0.4, 0.5]
    ],
    [
      [0.2, 0.3],
      [0.4, 0.5]
    ],
    [
      [0.7, 0.4],
      [0.2, 0.0]
    ]
  ]])



To check this, you could apply the logit function, log (p / (1 - p))
to convert your probs1 tensor, and then run that through
CrossEntropyLoss.
Best.
K. Frank