Thanks @richard for the .is_cuda() tip!
It’s strange when I tried to do is_cuda() on a torch.cuda.LongTensor variable:
>>> word_inputs[0]
Variable containing:
 1
[torch.cuda.LongTensor of size 1 (GPU 0)]

>>> type(word_inputs[0])
torch.autograd.variable.Variable

>>> word_input[0].is_cuda()
TypeError      Traceback (most recent call last)
---> word_inputs[0].is_cuda()
TypeError: 'bool' object is not callable

I get the same TypeError for the other inputs too:
>>> encoder_outputs.is_cuda()
TypeError      Traceback (most recent call last)
---> encoder_outputs.is_cuda()
TypeError: 'bool' object is not callable

>>> decoder_hidden].is_cuda()
TypeError      Traceback (most recent call last)
---> decoder_hidden.is_cuda()
TypeError: 'bool' object is not callable


>>> decoder_context.is_cuda()
TypeError      Traceback (most recent call last)
---> decoder_context.is_cuda()
TypeError: 'bool' object is not callable

Any clues?