In your case, it seems that there is no cost for them: as you said total runtime with CUDA_LAUNCH_BLOCKING=1 is the same as without, sothe async does not buy you a significant amount of time. So the place where the sync happens (everywhere, at the .item() or in the dataloader) will not impact the total runtime.