Hi,
I suggest you to follow this thread; it is active for months.

  
    
    
    Guidelines for assigning num_workers to DataLoader 
  
  
    Yeah I’ve since changed tune about convergence. Agree. That’s what I got when batch size was the same, so parallelisation was NOT working properly. Overheads of parallelisation don’t pay off without increasing batch size. Right?