Yes, I added it, and here is my code
import pdb
import time
import torch

x = torch.rand([30,30])
x = x.cuda()
y = torch.rand([30,1])
y = y.cuda()

torch.cuda.synchronize()
start = time.time()
for i in range(10):
  w = torch.gesv(y,x)
end = time.time()

print (end-start)/10.0

x = torch.rand([30,30])
y = torch.rand([30,1])

w = torch.gesv(y,x)

start = time.time()
for i in range(10):
  w = torch.gesv(y,x)
end = time.time()

print (end-start)/10.0

the test result is 0.00462810993195 on GPU and 2.31027603149e-05 on CPU.
This confused me a long time.@albanD