You have to use DistributedDataParallel with as many processes as the value of world_size. If you specify rank=1 on a single process, it will hang around waiting for the process with rank=0 to start.
Check out the launch utility 9 for easy launching of multiple processes. You can use it like this:
python -m torch.distributed.launch --nproc_per_node=2 ./my_script.py