@theevann @cognitiverobot
Short answer: You want synchronize() after end.record() as in the original post above.
Long answer:
The way this works is the function call and the events are added to a CUDA stream object, in order. Each kernel and event in a stream is executed in order, synchronously. The only asynchronous part (for a single stream) is with respect to the CPU. When you call torch.cuda.synchronize() it will wait for the GPU to complete all events in the (default) stream, before returning control to the CPU. So you want everything, including your end event, before the synchronization call. You can read more about it in a nice NVIDIA blog post here 95.