It’s probably not the error, but you should call .cuda on the Tensor before wrapping it in a Variable (for example in this case Variable(Y[i]).cuda().
Could you check the gradients with rnn.i2h.weight.grad?
Also could you provide the shapes of X and Y?