Thanks for the code!
nn.CrossEntropyLoss applies F.log_softmax and nn.NLLLoss internally, so could you please remove the F.log_softmax from your model or use nn.NLLLoss as the criterion, and rerun the experiment?