Pytorch, Optimization, Deep Learning, Artificial Intelligence, Tensorboard.
GatherBackward trace hints that a torch.gather operation may be involved, but where is it coming from and why is it causing a synchronization event? In our previous posts (e.g., here), we advocated using labeled torch.profiler.record_function context managers in order to pinpoint the source of a