It really just depends on how much CPU time it takes to load data to keep the GPU busy so ultimately it depends on your particular dataset, preprocessing steps, and the relative speed of CPU and GPU in your setup. A simple way to set num_workers is to keep increasing it until your GPU utilization (e.g., in nvidia-smi) no longer increases.