Pytorch, Deep Learning, Optimization, Tensorboard, Deep Dives.
Core utilization, using AMP lowers the GPU memory utilization freeing up more space to increase the batch size. The image below captures the training performance results following the AMP optimization and the batch size set to 1024: Results of AMP Optimization in the TensorBoard Profiler Overview