It is OK 
I am trying to implement this tutorial 13.
When there is more than 1 GPU, I run this:
model = nn.DataParallel(model, device_ids=range(torch.cuda.device_count()))
I also change hidden layer line
from:
model.hidden = model.init_hidden()
to:
model.module.hidden = model.module.init_hidden()
Whenever there is a tensor, I change it to .cuda(). I think it has no issue since it runs when there is 1 GPU. However, when I use 4 GPUs in the cluster environment, it says the forward function has this error:

torch/nn/parallel/_functions.py", line 11, in forward
raise TypeError(‘Broadcast function not implemented for CPU tensors’)
TypeError: Broadcast function not implemented for CPU tensors

Thank you for helping me out