I would recommend to start by profiling the code and narrowing down the bottleneck.
E.g. if the data loading is the bottleneck, the GPUs might be starving so increasing the batch size wouldn’t help much.
You could run a quick test by using synthetic data.