The concepts of world_size and rank are defined on processes (hence the name process_group). If you would like to create 8 processes, then the world_size should be 8, and the ranks for them should range from 0 to 7. It is up to the application to determine how to place processes to machines. In the above cluster (2 machines, and 4 GPUs each), the best setup would be creating 4 processes on each machine, with each exclusively working on a different GPU.