Given two matrices A and B, both square, my naive assessment would be that the multiplication A * B should be much slower than A * B^T, since in the first case the sum
C_ij = sum_j A_ij * B_ji

does not respect the row-major ordering of B whereas when I am taking the transpose of B it does. Nevertheless I cannot measure a significant difference:
a = torch.empty(5000, 5000).normal_()
b = torch.empty(5000, 5000).normal_()

start = timer()
torch.matmul(a, b)
end = timer()
print(end - start) 

start = timer()
torch.matmul(a, b.T)
end = timer()
print(end - start)

prints
0.7230800580000505
0.6886540420236997

Why is that so?