Hi Shirui!



 shirui-japina:

By chance I saw the code here.


Well, yes, as you have recognized, this code is wrong.


...
        return F.log_softmax(x, dim=0)
...
criterion = nn.CrossEntropyLoss()

My first question is, F.log_softmax (x, dim = 0) shouldn’t be used here, is my understanding right?


That is correct.  When using CrossEntropyLoss you should not use
log_softmax() for the output of your model.  (You would, if you were
using NLLLoss.)  You would typically pass the output of the last linear
layer of your model into CrossEntropyLoss (as you indicated in the
code you posted).
One more error in the code you linked to:
Even if it were appropriate to use log_softmax() (for example, with
NLLLoss), log_softmax (x, dim = 0) is wrong.  x here has shape
(nBatch, nClass), so log_softmax (x, dim = 0) would perform
the softmax operation across the batch dimension.  You would need
instead log_softmax (x, dim = 1) in order to perform softmax
across the class dimension.
(Unfortunately, there is a lot of misinformation on the internet.)
Good luck.
K. Frank