Solved by albanD in post #2 
              
              
                Hi, 
This happens because the CUDA api is asynchronous. So the first iteration, what you measure is just the time to queue work on the GPU. 
After a while, that queue is full and you have to wait for stuff to be actually done before being able to enqueue more work. 
You can use the equivalent of tor…