Pytorch, Optimization, Deep Learning, Artificial Intelligence, Tensorboard.
stages of the backward propagation. In this post, we have shown how PyTorch backward hooks, along with torch.profiler.record_function, can be used in an iterative process in order to identify the source of performance issues in the backward pass. We applied this technique to a simple ViT model and