I got this error when doing the backward pass:
RuntimeError: Function ThAddmmBackward returned an invalid gradient at index 1 - expected device 1 but got 0
This occurs when executing loss.backward(), where
ad_loss = torch.nn.BCELoss().cuda()
class_loss = torch.nn.CrossEntropyLoss().cuda()
a_loss = ad_loss(ad_out, ad_label)
c_loss = class_loss(s_output, s_target_var)
loss = c_loss + a_loss

The pytorch version is: 1.0.0a0+14004cb which is the current version on Master branch. The python version is: 2.7.15. And I am using 2 GPUs. Any idea is appreciated. Thanks in advance!