Solved by cbalioglu in post #2 
              
              
                torchrun is part of PyTorch v1.10. If you are running an older version,  python -m torch.distributed.run command serves the same purpose.