dist.get_rank() returns either 0 or 1 even though I launch the training with
python -m torch.distributed.launch \    --nproc_per_node=4 \ 	 --nnodes=2 \ 	 --node_rank=0 \ 	 --master_addr="$MASTER_ADDR" \ 	 --master_port="$MASTER_PORT" \    train_dist.py & 
I am using 2 nodes each with 4 GPUs.