It seems like the issue is #16433 44.
A workaround would be to calculate the gradient-penalty directly (without calling a function to do so) and calling backward in the same scope.
For example the following code will explode on CUDA with multi-gpu:
gp = calc_grad_penalty(network, real_target, fake_target)
gp.backwards(retain_graph=True)

While the following does not:
### gp += torch.autograd.grad()
### etc. etc. code to calcualte GP
gp.backwards(retain_graph=True)