Are you initializing the hidden state yourself? If you don’t manually initialize the hidden state, PyTorch will auto-initialize them for you (all zeros). This way, you wouldn’t have to mess around with shifting the hidden to cuda.
Side note: There’s an option in the editor called “Preformatted text”. Please use that to format your code to make it more readable.