@apaszke Thanks for the tip. Corrected times below, hopefully correct:
In [1]: import torch; torch.Tensor(1).cuda() # for cuda init
Out[1]:
1.00000e-36 *
  5.4583
[torch.cuda.FloatTensor of size 1 (GPU 0)]

In [4]: %timeit torch.mm(torch.Tensor(512, 20).cuda(), torch.Tensor(512, 20).cuda().t()); torch.cuda.synchronize()
The slowest run took 4.69 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 72.7 µs per loop

In [5]: %timeit torch.bmm(torch.Tensor(20, 512, 1).cuda(), torch.Tensor(20, 512, 1).cuda().transpose(-1, -2)); torch.cuda.synchronize()
10000 loops, best of 3: 135 µs per loop