In my case the issue appeared to be that the  dtype  of the initial hidden state was a double and the input was a float. I don’t quite understand  why  that is an issue, but casting the hidden state to a float solved the issue. If you have any intuition about why this might be a problem for PyTorch, do let me know in the comments.