Solved by ptrblck in post #2 
              
              
                I’m no expert in RNN, but it seems you are keeping the complete history of your hidden states, which uses more memory for each epoch and also slows down the code after a while. 
If you re-initialize your hidden states in each epoch with: 
model.gru_hidden1 = model.init_gru_hidden(batch_size)
model.g…