It might depend how you are using DistributedDataParallel.
The fastest and recommended way would be to create a process for each GPU, which would also mean that these process use their own DataLoaders.
For 8 GPUs (8 processes in DDP), and e.g. num_workers=2, you should see 8*2=16 processes.