It depends a bit where the current bottleneck is.
Using amp should reduce the execution time, if TensorCores can be used. However, increasing the batch size would also increase the time again. The net benefit depends on the achieved speedup through the usage of TensorCores vs. the increased workload.
That being said, increasing the batch size also increases the data loading time, since each worker has to load more samples now and your current setup might face a data loading bottleneck.
You could profile the data loading as shown in the ImageNet example 16 and check, if this time decreases during the training, which would mean that all workers can preload the batches in the background while the GPU is busy with the training.