Have you tried not using a DataLoader …? if the dataset is 1D, and sufficiently small, you could just preload it in the GPU. Then write a custom loader that directly uses indices into your TensorDataset , should significantly speed up your training epochs.