@leo-mao, you should not set world_size and rank in torch.distributed.init_process_group, they are automatically set by torch.distributed.launch.
So please change that to dist.init_process_group(backend=backend, init_method=“env://”)
Also, you should not set WORLD_SIZE, RANK env variables in your code either since they will be set by launch utility.