It’s a limitation from the cudnn implementation.
From the cudnn docs 4:

fwdMode Input. Specifies inference or training mode (CUDNN_FWD_MODE_INFERENCE and CUDNN_FWD_MODE_TRAINING). In the training mode, additional data is stored in the reserve space buffer. This information is used in the backward pass to compute derivatives.

Thus if you call model.eval(), you won’t be able to calculate the gradients using the cudnn RNN implementation. As described before, calling:
model.eval()
model.rnn_layer.train()

could solve this issue, as it would keep the RNN in training mode.