Hey @mrshenli
Thanks so much for you reply. What are the other sync points in DDP? I remember reading somewhere here that copying data from the gpu to the cpu forces a sync. Is this true? And does it still happen in a no_sync context?
Once again, thanks for your time.