The JIT is optimizing the model and its graph in the first iterations, which would add some overhead to them. Generally, the first iteration would also see an overhead from the cuda memory allocation (which would then be cached) etc.