Solved by ptrblck in post #2 
              
              
                nn.NLLLoss expects log probabilities so you should use F.log_softmax as the last activation instead of F.softmax. Could you change it and check, if your training performs better?