I am actually seeing some speed up, just not as much as I expected…
Currently I when testing Preloading data onto GPU (slightly faster than parallel data loading I imagine but less scalable). I am getting a time reduction from 42 secs → 25 secs per epoch (thanks to your help!).
That’s almost 2x speed with x4 GPUs, it’s not bad but I’m wondering if I should be seeing more?
P.S. I am seeing 100% GPU utilization on each GPU in spurts but not consistently.