While training a model when i use multi-gpus i get this error.
loss.backward()
  File "/home/ashwin/anaconda3/lib/python3.6/site-packages/torch/autograd/variable.py", line 120, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/ashwin/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py", line 81, in backward
    variables, grad_variables, retain_graph, create_graph)
RuntimeError: arguments are located on different GPUs at /home/ashwin/pytorch/aten/src/THC/generated/../generic/THCTensorMathPointwise.cu:231


I just use model.cuda() and similarly while loading variables i use variable(<param_name>).cuda()
and while training in single gpu it is working fine but while multi-gpu it is giving this error. Any help in this is appreciated .
I could also see torch.cuda.device_count() = 4 and torch.cuda.current_device()=0