From this log message: ptwop-176:1755:1869 [0] include/socket.h:390 NCCL WARN Connect to 162.105.146.178<54473> failed : No route to host. I’m assuming the other node’s ip is 162.105.146.178? Could you validate the following:

See if the issue reproduces on a single-node multi-gpu setup.
Can you ping 162.105.146.178 from 162.105.146.176?