If you didn’t change any hyperparameters, I assume the batch size is also the same as for the single GPU run.
If that’s the case, your training might suffer, if you are using batchnorm layers, as their running estimates might be off due to the lower batch size on each device.
DistributedDataParallel provides SyncBatchNorm, which synchronizes these stats between the processes.
PyTorch 1.1.0 is also not really new. Are you having trouble updating to the latest version?
If so, could you post the issues, so that we can help with it?