Solved by Ra-V in post #4 
              
              
                I tried using nn.DataParallel, but I ran into the same problem. The diagnostics of GPUs looks like this: 

Sun Jan 26 12:16:30 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.01    Driver Version: 418.87.01    CUDA Version: 10.1     |
|…