Right, learning rate, batch size, loss function can all play a role here. For example, if using the same per-process batch size, then the DDP gang collectively will consume more samples with larger world size, and hence the learning rate will need to adjust accordingly.