Thanks, the code started running, no longer stuck at the distributed barrier but the nvidia-smi output still doesn’t make sense to me. Two processes are running on both the GPUs.

Is this the expected behavior or something’s wrong?