Hi thanks for your input.
ifconfig returns me multiple interfaces. I picked the one that has the IP address assigned I use to login (10.10.10.22 and 10.10.10.25).
First machine:
br-5443622090a7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
        inet6 fe80::42:1aff:fec8:448c  prefixlen 64  scopeid 0x20<link>
        ether 02:42:1a:c8:44:8c  txqueuelen 0  (Ethernet)
        RX packets 178271  bytes 4991588 (4.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 39  bytes 5694 (5.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:e5ff:fe0a:b382  prefixlen 64  scopeid 0x20<link>
        ether 02:42:e5:0a:b3:82  txqueuelen 0  (Ethernet)
        RX packets 76931  bytes 4131399 (4.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 86761  bytes 625271597 (625.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.22  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::3eec:efff:fe03:ed46  prefixlen 64  scopeid 0x20<link>
        ether 3c:ec:ef:03:ed:46  txqueuelen 1000  (Ethernet)
        RX packets 206957342  bytes 291080003217 (291.0 GB)
        RX errors 0  dropped 3340746  overruns 0  frame 0
        TX packets 47119321  bytes 7465744617 (7.4 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xc1320000-c133ffff

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 31035316  bytes 3161891159 (3.1 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 31035316  bytes 3161891159 (3.1 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth212871c: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::e400:bff:feb2:acb0  prefixlen 64  scopeid 0x20<link>
        ether e6:00:0b:b2:ac:b0  txqueuelen 0  (Ethernet)
        RX packets 339096  bytes 21260368 (21.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 359220  bytes 21820500 (21.8 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth406ee85: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::d827:24ff:fe8c:3c0f  prefixlen 64  scopeid 0x20<link>
        ether da:27:24:8c:3c:0f  txqueuelen 0  (Ethernet)
        RX packets 1807  bytes 131066 (131.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2265  bytes 13311970 (13.3 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethb0ca500: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::384f:23ff:fe37:a7b9  prefixlen 64  scopeid 0x20<link>
        ether 3a:4f:23:37:a7:b9  txqueuelen 0  (Ethernet)
        RX packets 338768  bytes 21241184 (21.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 359348  bytes 21826112 (21.8 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Second machine:
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:e5ff:fe55:c1a2  prefixlen 64  scopeid 0x20<link>
        ether 02:42:e5:55:c1:a2  txqueuelen 0  (Ethernet)
        RX packets 990861  bytes 50900887 (50.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1872198  bytes 4152860383 (4.1 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp49s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.25  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::3eec:efff:fe1e:dd5b  prefixlen 64  scopeid 0x20<link>
        ether 3c:ec:ef:1e:dd:5b  txqueuelen 1000  (Ethernet)
        RX packets 82711207  bytes 110825052923 (110.8 GB)
        RX errors 0  dropped 489968  overruns 0  frame 0
        TX packets 25754173  bytes 2401860983 (2.4 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 36992814  bytes 2445844459 (2.4 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 36992814  bytes 2445844459 (2.4 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Tried setting NCCL_SOCKET_IFNAME=eno1, enp49s0f1 on both servers but it didn’t help unfortunately.
I replaced the barrier with an allreduce like so:
    x = torch.tensor([args.global_rank], dtype=torch.float, device=torch.device("cuda", 0))
    torch.distributed.all_reduce(x)
    print(x)

but it hangs the same way as with barrier.
The two network interfaces can talk to each other, I verified that I can listen on one machine and send a message through telnet to the other machine:
# on one server
nc -l 1191
# on the other server
telnet 10.10.10.22 1191

and this works both ways.