Please check the doc here 1 for synchronize.
In particular, if you want to check how long a given net takes to run on gpu, you need to do:
torch.cuda.synchronize()
start = time.time()
out = net(in)
torch.cuda.synchronize()
end = time.time()
elapsed_time = end - start

This makes sure that nothing was still runing on the gpu when you start and then waits for your stuff to run before measuring the end time.