Increasing the batch size would also increase the overall device utilization assuming you are not creating a data loading bottleneck, but since I’m not familiar with your model I don’t know how much improvement an increase in the batch size would give. You could profile your code and check the improvement via the native PyTorch profiler or e.g. Nsight Systems.