Is this on mobile or on server? I know there is some work being done on the speed of add and mul here: AVX512 and Vec512 · Issue #56187 · pytorch/pytorch · GitHub 10
if its on mobile, it may be necessary to involve the mobile team as well.