Hi it’s usually simpler to start several python processes using the torch.distributed.launch utility of PyTorch. Here 1.1k is a (very) simple introduction about distributed training in PyTorch (there are several ways you can improve over that but it will show you an example in action).