Thank you very much! This is really the answer to the difference! After setting pin_memory=False, the performance of PyTorch 1.2 with CUDA 9 and of the one with CUDA 10 were the same. I didn’t test 1.0 and that’s really interesting to see 1.0 is even faster.