It depends on the utilization of the GPU and how/if the operations are already parallelizing in other dimensions than the batch size, such as the spatial dimensions of your input tensors.
If your GPU utilization is already high, you won’t see a linear speedup.
While training, you will of course use less iterations per epoch, which is another speedup, but unrelated to your use case.