Thank you for your reply. OK, now, I write this test code like this
import pdb
import time
import torch

x = torch.rand([30,30])
x = x.cuda()
y = torch.rand([30,1])
y = y.cuda()

torch.cuda.synchronize()
start = time.time()
w = torch.gesv(y,x)
end = time.time()

print (end-start)

x = torch.rand([30,30])
y = torch.rand([30,1])

w = torch.gesv(y,x)

start = time.time()
w = torch.gesv(y,x)
end = time.time()

print (end-start)


the result stills 0.00907206535339 s on GPU and 5.60283660889e-05s on CPU, why?