Same situation.
I have 12 cores, I find that the dataloader can not make full use of them. It can only make about 60% ability of them. I set the num_worker as 12 and my IO bound is enough.  For training network like resnet, the data preprocess time cost more than 70% time in the whole process.