That is weird indeed. Can you isolate the problem to data loading (i.e. don’t train a model, just iterate over the data set)? Due to asynchronous nature of CUDA, wall clock time that attribute to data transfer is in fact caused by asynchronous execution of for example autograd, the optimizer, etc.