So I read up a little and it seems that recurrent dropout is not yet implemented in PyTorch, and there is an open Github issue 4 for the same. I guess the only solution is to manually write the LSTM cell then.
However, shouldn’t absence of such a dropout be responsible for overfitting at max?
I still can’t wrap my head around the fact that this is the sole reason why the model isn’t learning at all (neither the training nor the dev loss seem to decrease at all).