Oh - and I should say. I’m not hung up about the tics and tocs - those were just for demonstration purposes to profile how the time per iteration changes throughout the loop. The important thing is the total wall time to process a bunch of images.
In production, I’d like this to infer around 100 512x512 images at a time. If this took sub-1s total (i.e., 100*0.002s as in the first few iterations in my first post), this would be great, and even sub-10s would be fine, but 45s is not really viable.