Hi,
I would like to illustrate the speed of tensor operations on GPU for a course.
The following piece of code:
x = torch.cuda.FloatTensor(10000, 500).normal_()
w = torch.cuda.FloatTensor(200, 500).normal_()

a = time.time()
y = x.mm(w.t())
b = time.time()
print('batch GPU {:.02e}s'.format(b - a))

a = time.time()
y = x.mm(w.t())
b = time.time()
print('batch GPU {:.02e}s'.format(b - a))

prints
batch GPU 1.06e-01s
batch GPU 3.43e-04s

so I presume that there is some “lazy operations” delayed until the first mm, and that the first timing includes some memory allocation or copy or something.
Is that the case? If yes, is there a proper way to force all this to be done, some sort of cuda.flush()?
Cheers,