As mentioned here: http://pytorch.org/docs/notes/multiprocessing.html#sharing-cuda-tensors 573
If you want to use CUDA, you have to use the forkserver or spawn start methods.
DataLoader with anything greater than 0 workers uses multiprocessing.