PyTorch would synchronize the code (for you), if a CPU operation is used. If I’m not mistaken a Python print would fit this definition. However, besides that you should still be careful about your custom CUDA code as you can create race conditions by missing necessary syncs.