I got the same issue here…
temp.shape
Out[14]: torch.Size([1140, 1140])
which is low rank structure
u,s,v = torch.svd(temp)
it took 3.5 seconds!
vs
u,s,v = torch.svd(temp.cpu())
it took 0.23 seconds!
ridiculously slow here…