You can use
torch.cuda.synchronize()
start = time.time()

everywhere there is a time.time() call to avoid using events. Can you check what is the behavior of this when you run on gpu? Does it give the same result as cpu or the same result as with events?