The implementation looks indeed wrong, as the code seems to combine nn.LogSoftmax with nn.CrossEntropyLoss.



 kirk86:

I’ve read somewhere that nn.CrossEntropyLoss() implicitly applies nn.LogSoftmax on the output from your net, is that true?


Yes, you can see it in this line of code 54.



 kirk86:

I’ve also read that if you want to be more verbose you could use nn.NLLLoss() with nn.functional.F.log_softmax() , is that true? In some experiments with some small MLPs using the above combination didn’t yield as good results as using simiply nn.CrossEntropyLoss() , are there any other intrinsic differences we should be aware of?


Both approaches should yield the same results.
The only possible mistake I could think of is if you specify the wrong dim using F.log_softmax.