Performance, Pytorch, Deep Learning, Artificial Intelligence, Tensorboard.
Our step time has dropped all the way down to 5.4 milliseconds. What happened?!! By simply changing around a few function calls and without any modification to the loss function logic, we were able to remove the synchronization points from the training step. Importantly, the average step time when