Ganga:

I have confirmed that he is using softmax in cross entropy loss


This would be wrong, since nn.CrossEntropyLoss expects raw logits and applies log_softmax internally as described in the docs 1.