slavavs:

Is there a tutorial to speed things up on 8 GPUs without increasing the batch size?


Why would want to keep the global batch size constant and split the computation to different devices?
Note that GPUs will usually achieve a higher utilization for a larger workload (which would e.g. correspond to the batch size).