I haven’t used both  nvprof/nsys and torch.cuda.synchronize().
I have just used PyTorch’s autograd profiler with use_cuda = True and torch.autograd.profiler.record_function() over these functions.
Is that not the right way to profile in PyTorch? Also, I didn’t understand the use of nvprof for CUDA events. Can you explain about it?