Solved by Eta_C in post #2 
              
              
                I think most of the time is spent sending data to the GPU, instead of calculating multiplication on GPU. 
import torch

a = torch.randn(1000000, 3)
b = torch.randn(1000000, 3)

with torch.autograd.profiler.profile(use_cuda=True) as prof:
    torch.mm(a.T, b)
print(prof.key_averages().table(sort_by="…