Hi everyone - super quick question. I looked around and I couldn’t find this previously asked, but my apologies if I missed something!
Wondering if I have HF trainer set up using torch.distributed.launch on 8 gpus… if my dataloader_num_workers = 10… is that 10 total processes for dataloaders or 10*8=80 processes?
Thank you again!