@ptrblck Actually I have a lot RAM in excess (more than 300 GB). So, that cannot be the case. Also, I tried batch size=1, still didn’t work for the 3 GPU run.
Another thing I noticed: when I set num_workers=0 in DataLoader, it works perfectly for the 3 GPU run. But setting num_workers=0 increases the training time drastically so I don’t like it. Is there an alternate fix (i.e. to use 3 GPUs while keeping my num_workers>0)?