You need to put a torch.cuda.synchronize() before every call to time.time() to get actuall timings. Otherwise you’ll just see random artifacts from the time it takes to queue stuff on the GPU and when that queue is full.