I’m no expert in RNN, but it seems you are keeping the complete history of your hidden states, which uses more memory for each epoch and also slows down the code after a while.
If you re-initialize your hidden states in each epoch with:
model.gru_hidden1 = model.init_gru_hidden(batch_size)
model.gru_hidden2 = model.init_gru_hidden(batch_size)

the memory usage and speed is constant.