Running into this problem as well, using a WGAN-GP and it works perfectly on 1 GPU but the loss explodes when running on multiple GPUs.
Using CUDA 9.2, PyTorch 1.0.1. Working on installing the nightly build to see if there is any difference.