Yes, the nodes’ ips are 162.105.146.178 and 162.105.146.176.

The issue cannot reproduce on a single-node multi-gpu setup, and everything runs well.
The two nodes can ping each other successfully, and it runs well when I change the communication backend to “gloo” from “nccl” with nearly no code changed (minus some modification to make Tensors and model on cpu).