Hi,
The GPU api is asynchronous. To get accurate measurement, you have to use torch.cuda.synchronize() before each time.time() call.
Also if you have cudnn, you can use torch.backends.cudnn.benchmark=True to get the very best performance (at the cost of the first forward being slower).