Hi,
I encountered the similar problem for DataLoader.
Sorry to ask the similar question, after reading all your discussion, I am still confused about the relationship between the number_GPU, num_CPU and the num_works.
Here is my situation:
I have a convolutional Autoencoder (so it’s not a very big network), but I have a very big dataset: For one epoch, I have `

16 (batch_size) * 7993 = 12788

images, each image’s dimension is 51 x 51 x 51.
So I used one GPU (Tesla P100) and set the num_workers=8. I also tried other options for num_works, like 0 or 16.
Always, it is very slow to load the data, the training time for each batch is very fast. To load each image, I found that with num_workers=8, it will take longer than num_workers=0. Every time, I will take a long time to load 74 batches and run the training, then reload another 74 batches (the case of num_workers=8).
Do you have any suggestions to set the num_workers if the dataset is huge???
Thanks in advance