Optimization, Pytorch, Machine Learning, Deep Learning, Tensorboard.
kernel fusion and out-of-order execution to map the loss function into low-level compute kernels in a manner that is optimal for the underlying training accelerator. criterion = torch.compile(torch.nn.CrossEntropyLoss().cuda(device)) The image below shows the Trace View result of this experiment