Hi @mrshenli,
I was looking at the tutorial 3 you mentioned.
In the example, it says that

This example uses a  torch.nn.Linear  as the local model, wraps it with DDP, and then runs one forward pass, one backward pass, and an optimizer step on the DDP model. After that, parameters on the local model will be updated, and all models on different processes should be exactly the same.

I’m just wondering what it means by local model, and what’s the difference between the local model and the models on different processes.
Thanks!