In that case the error might be raised, if you create CUDATensors in your Dataset.
The vanilla use case would be to create CPU tensors in the dataset, process them, and push them to the GPU inside your DataLoader loop.