I assume that you are working on a multi-class classification and apply nn.CrossEntropyLoss as your criterion.
If that’s the case, then you should remove the softmax from your model, as nn.CrossEntropyLoss expects raw logits and will apply F.log_softmax and nn.NLLLoss internally.
Your additional softmax would reduce the gradient magnitude which would explain the slow learning of your model.