Just to add some more insight to this, we have a bucket_cap_mb argument in the DDP constructor. This defines the size of a gradient bucket in megabytes. During the backward pass, each rank fills the bucket with gradients and then kicks off the allreduce collective. Faster ranks will kick off the allreduce collective earlier than the slower ranks, so they will just block until the slower ranks kick off the collective. No gradients are abandoned in this process. You can tune the bucket_cap_mb as desired, and this will trigger allreduce more frequently for smaller buckets and less frequently for larger buckets.
If the performance difference is too great, you can explore syncing gradients less frequently (every n batches instead of every batch) using the model.no_sync() context manager or using multiple process groups (using the new_group API). If you find DDP training getting stuck due to excessively long hang-times due to these blocked collectives, you may look into using torchelastic and some mechanism to timeout hanging collectives (such as NCCL_ASYNC_ERROR_HANDLING or NCCL_BLOCKING_WAIT - docs here 4)