You don’t need to synchronize your code for training.
It’s just necessary, if you would like to time your code.
CUDA calls are asynchronous, i.e. they work in the background while your python code calls the next line, stopping the time.
That is why you need to wait for the CUDA operation to finish, before you take the time.
In your live training code, you should remove the synch.