I don’t know what your use case and in particular loss function is, but the last softmax operation is often wrong. In case you are working on a multi-class classification use case using nn.CrossEntropyLoss, note that this loss function expects raw logits so you would have to remove the x = x.softmax() call.
Also, make sure to add the weight decay to the optimizer for the same parameters as was done in TF.