Do you mean I need to running instances of this code in different processes?

Yes, you can refer to these docs for concrete examples: Writing Distributed Applications with PyTorch — PyTorch Tutorials 1.8.1+cu102 documentation 22