Check out these resources. They helped me understand how to do it. I agree that the docs as of now are not to the point.

Distributed data parallel training in Pytorch 3
GitHub - GoldenRaven/Pytorch_DistributedParallel_GPU_test: Pytorch distributed data parallel test of GPU on MNIST 1
https://towardsdatascience.com/how-to-convert-a-pytorch-dataparallel-project-to-use-distributeddataparallel-b84632eed0f6 2
Distributed Training in PyTorch (Distributed Data Parallel) | by Praneet Bomma | Analytics Vidhya | Medium 1