Yep, here is a starter example: Distributed Data Parallel — PyTorch 1.8.0 documentation 3
You can replace the torch.randn(20, 10).to(rank) random input tensor by input and labels from a dataloader example 1
Here is a complete list of DDP tutorials: PyTorch Distributed Overview — PyTorch Tutorials 1.8.0 documentation 1
It should work, but will have different accuracy implications to different applications, depending on your model, data, batch size, loss function, etc. I would suggest taking a look at the DDP paper or at lest the design notes and verify whether DDP’s gradient averaging algorithm is OK for your application.