Hi all,
I am trying to measure the speedup which can be obtained in alexnet training by exploiting multiple GPUs.
I am using this code https://github.com/pytorch/examples/blob/master/imagenet/main.py 62 without any modification. The training set is taken from imagenet (Large Scale Visual Recognition Challenge 2012) and is composed of 100 classes with 1000 images. Batch size is 1024, Number of epochs is 10. I am running the same code with the same parameter and inputs on Microsoft Azure NV6 (1 x NVIDIA Tesla M60) and NV12 (2 x NVIDIA Tesla M60). Installed pytorch is 0.3.1 (with CUDA 9.0).
I would expect that code runs faster on 2 GPUs (not 2x but at least a significant speedup), but on the contrary, I am obtaining the same execution times, around 3500 seconds.
Is there any parameter which I am not correctly setting or something that I have to modify in order to actually exploit all the available GPUs?
Thanks