There is no such a think like multi-gpu dataloader.
The most you can do is to use dataloader as datareader.
Then set a nn.module for preprocessing. Apply data parallel over it and do any preprocessing you need.
I doubt it were efficient but you can try. You should have a very dense preprocessing to feel the advantage of spliting the data in several gpus.
Another option is you to use a single gpu for preprocessing and another one for the main workload.