ptrblck:

before starting and stopping the timer and compare the results


thank you ! now when timing the code, every iteration takes the same amount of time,
but there is one thing i don’t understand, if i do not synchronize cuda, when i reach the line
print(time() - t1)

all the computations must be complete, as i already called backward and optimizer.step and so on …
in order to reach this line, all the computations must be complete, so how come it is sometimes faster and sometimes slower ?
is the pytorch code evaluated lazily ? meaning if i call some cuda operation before another is complete it is added to some queue ?