Hi @mrshenli,
I guess I found the problem. If I do export GLOO_SOCKET_IFNAME=nonexist in the terminal then it does not become an environment variable in the Jupter Notebook. But see it in Python launched from that terminal directly.
So I guess I have to do the other way around and set that in the JupyterNotebook explicitly? Here is the result if I do what you suggested.



 mrshenli:

can you keep/uncomment the init_method line or set MASTER_ADDR and MASTER_PORT ? It seems failed during Python land arg checking due to missing master addr/port, before entering C++ pybind methods.


import torch.distributed as dist
import os
​
print(os.environ.get('GLOO_SOCKET_IFNAME'))
​
os.environ['MASTER_ADDR'] = 'localhost'
os.environ['MASTER_PORT'] = '23456'
​
os.environ['GLOO_SOCKET_IFNAME']='nonexist'
print(os.environ.get('GLOO_SOCKET_IFNAME'))
# on rank 0
dist.init_process_group(
    backend = "gloo",
    init_method = 'tcp://10.1.1.101:29500',
    rank = 0,
    world_size = 1
)
​
​
​
None
nonexist
----------------------------------------------------------------------
RuntimeError                         Traceback (most recent call last)
<ipython-input-1-ad5d77a63395> in <module>
     14     init_method = 'tcp://10.1.1.101:29500',
     15     rank = 0,
---> 16     world_size = 1
     17 )
     18 

~/anaconda3/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py in init_process_group(backend, init_method, timeout, world_size, rank, store, group_name)
    401             store,
    402             group_name=group_name,
--> 403             timeout=timeout)
    404 
    405     _pg_group_ranks[_default_pg] = {i: i for i in range(_default_pg.size())}

~/anaconda3/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py in _new_process_group_helper(world_size, rank, group_ranks, backend, store, group_name, timeout)
    469                 rank,
    470                 world_size,
--> 471                 timeout=timeout)
    472             _pg_map[pg] = (Backend.GLOO, store)
    473             _pg_names[pg] = group_name

RuntimeError: [enforce fail at /opt/conda/conda-bld/pytorch_1587428398394/work/third_party/gloo/gloo/transport/tcp/device.cc:83] ifa != nullptr. Unable to find address for: nonexist

Thanks,
Ziyi