Tony_Lee:

File “demo.py”, line 58, in 
x = sum([tensor * w for tensor, w in zip(tensors, probs_i)])


Seems to be to blame here. But if the elements are on different GPU, it should fail during the forward, never the backward.
After investigation, a minimal repro is:
import torch

torch.autograd.set_detect_anomaly(True)

a = torch.rand([], requires_grad=True, device="cuda:0")
b = torch.rand(10, requires_grad=True, device="cuda:1")

c = a * b
c.sum().backward() # Fails with the same error

I’ll open a github issue for that as soon as github is back online 
To fix your code, you should make sure that every element is on the right device. In particular, here, it seems like probs is not on the right GPU after giving it to the DataParallel. a simple fix would be probs_i = probs_i.to(tensors[0].device).