Hi Shen,
I always set the CUDA_VISIBLE_DEVICES for each run using export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 for example. The code runs on all the 8 GPUs with full utilization, so multiprocessing is surely working. The command I use is as follows  on the two servers I’m using (with appropriate IP and port set):
python train_distributed.py --dist-url 'tcp://ip:port' --dist-backend 'gloo' --multiprocessing-distributed --world-size 2 --rank 0
python train_distributed.py --dist-url 'tcp://ip:port' --dist-backend 'gloo' --multiprocessing-distributed --world-size 2 --rank 1