My code runs so much faster on my single GTX 1060 than on the cluster which has 2 GTX 1080 Ti.  This is strange because a few parts of the code run faster on the cluster, although most parts run slower.  For some comparison, enumerating through the dataloader takes ~9 seconds on my local while it takes 550 seconds on the cluster.  Also, calculating loss + backprop takes ~ 1 seconds on my local, while it takes 10 seconds on the cluster.  The code is from this paper https://github.com/Philip-Bachman/amdim-public 1