I’ve tried reimplementing a simple GRU language model using just a GRU and a linear layer, the code is at https://www.kaggle.com/alvations/gru-language-model-not-training-properly 19
But when the model is predicting, we see that it’s only predicting “the” and comma “,”.
Anyone spot something wrong with my code? Or hyperparameters?