I see. I wonder what is up. Can you still use your DataLoader, but make it return CPU Tensor, but cast it to CUDA and wrap it in a Variable inside the training loop?
I wonder if enumerate (or the iterator) queue up future iterations ahead of time.