I had the same error when I moved from Keras to Pytorch. I had a Keras model with Dropout like you (p=0.4, p=0.5), the model was learning correctly and I switched to Pytorch with the same data, same model and same Dropout value, and the model wasn’t learning, the loss didn’t decrease.
So I removed the dropout and that was working well.
I didn’t seek why there is a huge difference with Dropout.