Thank you very much for your quick and patient reply.
I measure these timings on GPU which is GeForce GTX 1080Ti.
If I want to measure these timings for a few part of my code, do I need to put  torch.cuda.synchronize()  at every place I do the measurement or do I just need to put 1 torch.cuda.synchronize()  at the outmost part?