Solved by mrshenli in post #6 
              
              
                Yep, here is a starter example: Distributed Data Parallel — PyTorch 1.8.0 documentation 
You can replace the torch.randn(20, 10).to(rank) random input tensor by input and labels from a dataloader example 
Here is a complete list of DDP tutorials: PyTorch Distributed Overview — PyTorch Tutorials 1.8.…