It’s not uncommon for the first pass to take a longer time. Since you have already preprocessed the data that’s not affecting the time in this case. The forward pass has some kind of setup code I’m guessing. It’s a bit weird that 1 image takes the same amount of time as 25. If you run 25 forward passes on separate images that should take a longer time than one forward pass on 25 images because the GPU can take advantage of matrix multiplications which are faster.
Maybe your model is just so tiny so the overhead of starting the inference dominates the time it takes to actually perform the computations?
In conclusion I don’t know why you are getting the results you are getting but it doesn’t seem like a big problem or abnormal to me. This library snakeviz 21 can help you get a bit more insight in your code execution time