actually when I do a print of x.get_device() and y.get_device(), where x and y are the tensors involved , they show the same device. Even then I get error, tensors are on different GPUs (using nn.Dataparallel). This occurs even for a vanilla resnet with some minor modifications.