Solved by ptrblck in post #3 
              
              
                I’m not sure about the TimestampCaptureCallback approach, but in any case you could add manual synchronizations via: 
auto stream = at::cuda::getCurrentCUDAStream(); // or at::cuda::getDefaultCUDAStream())
AT_CUDA_CHECK(cudaStreamSynchronize(stream));

to make sure your timings are correct. Alternat…