Solved by richard in post #2 
              
              
                There are many things you can do CPU-only benchmarking: I’ve used timeit as well as profilers. 
CUDA is asynchronous so you will need some tools to measure time. CUDA events are good for this if you’re timing “add” on two cuda tensors, you should sandwich the call between CUDA events: 
start = torch…