Can anyone point me to some recent performance profiling numbers for PyTorch training (e.g. which modules occupy the most CPU/GPU time during training, I am assuming that is autograd?)
Or more to the point. I am interested in implementing non-CUDA acceleration for training in PyTorch; could that be accomplished just with a custom activation and forward/backwards function?
Also, has there been any work towards AVX-512 CPU acceleration with PyTorch?
Thanks in advance