Olivier-CR:

is my understanding correct that in order to avoid CPU contention, num_workers should be <= cpu_count / 8 ?


Yes this is correct if your dataloader is CPU intensive, if the dataloader is IO intensive, you could have more num_workers to load data from the filesystem/network with a higher degree of parallelism (until you saturate the filesystem/nw bandwidht).