Thanks for the code!
I could reproduce the timing difference.
Unfortunately, I don’t know the exact reason for this. I couldn’t find e.g. if Kahan summation is used in PyTorch or any other linear algebra library. Let’s wait for someone more familiar with the implementations.