Yeah, I would avoid (2). Accessing the GPU from dataloader workers is the path to ruin.
You don’t have to do all your preprocessing in the dataloader. For example, you can do your file load and CPU pre-processing in the data loader, but do the GPU operations afterwards:
for sample_cpu in dataloader:
   sample_gpu = preprocess_gpu(sample_cpu)
   train(sample_gpu) # or whatever

In general, the way to make GPU operations fast is:

batch operations
avoid CPU-GPU synchronizations
make sure the underlying ops are efficient