Hi folks,
I’m bit confused in regards to the proper usage of cross entropy loss and log_softmax.
I’ve read somewhere that nn.CrossEntropyLoss() implicitly applies nn.LogSoftmax on the output from your net, is that true?
In that case is the implementation here 26 wrong?
I’ve also read that if you want to be more verbose you could use nn.NLLLoss() with nn.functional.F.log_softmax(), is that true? In some experiments with some small MLPs using the above combination didn’t yield as good results as using simiply nn.CrossEntropyLoss(), are there any other intrinsic differences we should be aware of?
Thanks!