qcyza:

I have torch.cuda.synchronize() at the beginning of the script. Is this what you refer to?


Yes, but it would be needed before starting and stopping all timers.
Otherwise your profile will accumulate the times in the next blocking operation and will yield wrong results.