For imagenet, for converged model accuracy, it entirely depends on the total batch size of the model training. In other words, the largest number of ResNet batch size without model accuracy loss I personally have done is 4096 images, even though other people have published about going as large as 18-24K batch size.
So imagine, if you put 32 batch size per GPU, with 4096 images, you will be distributing your distributed job to 128 GPUs.  From my own experience, from pure HPC performance and scalability point of view, I have done 64 nodes * 8 GPU/node  (512 GPUs) training myself with pretty good linear scalability. Of course, the scalability depends upon your model size, your HPC cluster network setup.
So what is your network interconnect, Infiniband or Ethernet? at what speed and what model are you trying to train?