The trick here is that there is a drop-replacement library of PIL I don’t recall the name right now. It’s pillow sidm or ssdm or something similar. That library is compiled and it optimizes the performance depending on the instructions available for ur cpu.
One of the drawbacks of the pytorch’s dataloder is that it cannot run multiprocessing together with gpu processing. That makes the answer a bit hard. It’s a trade-off which depends on the dataset, which GPU you have, which cpu… I would say that a good practice (at least to me) is trying to split the dataloading in a cpu-intensive + a gpu-intensive stages.
If there is something which is really worth to run in the GPU, for example, stft or really heavy ops, I just apply them after the batch is loaded.
But once again, everything depends on your computer, the type of data ecetera