Hi, what is the output of ifconfig on both your machines? If you’re using the ifconfig output to set the NCCL_SOCKET_IFNAME variables on each node, you could try setting NCCL_SOCKET_IFNAME=eno1, enp49s0f1 as per the comments on this issue: How to set NCCL_SOCKET_IFNAME · Issue #286 · NVIDIA/nccl · GitHub 2, and in general make sure that the two interfaces can talk to each other according to your network setup.
In addition, can you try changing barrier() to allreduce some tensors (allocated on the appropriate GPU) and check whether that works as expected?