Unfortunately, timing GPU functions like this can be unreliable because kernel launches are asynchronous. To get a more reliable timing measurement of the time per iteration, you can try adding a torch.cuda.synchronize() before timing stops. (Note that you probably want to remove this before actually running a long training job, as this can reduce performance)