Interesting, let’s first see if the code runs without cudnn. Could you try torch.backends.cudnn.enabled=False? This runs the RNN code that isn’t from cudnn. It will be slower compared to cudnn, but is a good thing to try and see if there is anything wrong with our inputs.