Solved by mrshenli in post #2 
              
              
                As batch_load measures the latency of executor.submit, I assume that’s the overhead of  ProcessPoolExecutor? 
But it is still weird that the optimize() also increased a lot. Does optimize() run ops on GPU? If yes, you will need to either torch.cuda.synchronize() on that GPU, or use elapsed_time to m…