Solved by ArneNx in post #3 
              
              
                The driver version is 440.82 and CUDA version 10.2. 
I think the drivers are the problem, since I had the same problem on any machine that uses these drivers (some of them have V100s instead of 1080s). 
However, I was able to get multi-GPU training running with the same driver version on another mac…