You can see the short note in the docs here 6 for why Distributed Data Parallel can be faster for multi-GPU training.
Yes, switching HDD to SSD can make a large difference, especially for “random” file I/O as loading many ImageNet images can look like random reads to storage. If you have sufficient memory, you might consider increasing the prefetch_factor in your DataLoader as well to see if increased buffering might help. However, if your HDD cannot keep up with the data loading speed, it is tricky fully utilize the GPUs.