Because in many cases, the loss function computes per-sample loss (e.g., default MSELoss) instead of aggregated loss (e.g., default sum). So, DDP calculates the average and tries to keep the gradients scale consistent with local training.