Solved by ptrblck in post #2 
              
              
                You could see the timing relatively between the CPU and GPU workload. 
E.g. if the GPU workload is small (due to a small model), you might see the overhead of the kernel launches as well as other CPU operations such as list(...), torch.stack(...), data loading, etc. 
To isolate the actual runtime of…