Thanks! I run this example (I change ‘‘gloo’’ to ''nccl") and get outputs:
(pytorch-env) wfang@Precision-5820-Tower-X-Series:~/tempdir$ NCCL_DEBUG=INFO python -m torch.distributed.launch --nproc_per_node=2 w1.py
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
Traceback (most recent call last):
  File "/home/wfang/tempdir/w1.py", line 36, in <module>
    main()
  File "/home/wfang/tempdir/w1.py", line 30, in main
    mp.spawn(example,
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
    while not context.join():
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 150, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException: 

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
    fn(i, *args)
  File "/home/wfang/tempdir/w1.py", line 11, in example
    dist.init_process_group("nccl", rank=rank, world_size=world_size)
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py", line 500, in init_process_group
    store, rank, world_size = next(rendezvous_iterator)
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/rendezvous.py", line 190, in _env_rendezvous_handler
    store = TCPStore(master_addr, master_port, world_size, start_daemon, timeout)
RuntimeError: Address already in use

Killing subprocess 3457045
Killing subprocess 3457046
Traceback (most recent call last):
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/wfang/anaconda3/envs/pytorch-env/bin/python', '-u', 'w1.py', '--local_rank=1']' returned non-zero exit status 1.
(pytorch-env) wfang@Precision-5820-Tower-X-Series:~/tempdir$ /home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 4 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

If I still use “gloo”:
(pytorch-env) wfang@Precision-5820-Tower-X-Series:~/tempdir$ python -m torch.distributed.launch --nproc_per_node=2 w1.py
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
Traceback (most recent call last):
  File "/home/wfang/tempdir/w1.py", line 36, in <module>
    main()
  File "/home/wfang/tempdir/w1.py", line 30, in main
    mp.spawn(example,
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
    while not context.join():
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 150, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException: 

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
    fn(i, *args)
  File "/home/wfang/tempdir/w1.py", line 11, in example
    dist.init_process_group("gloo", rank=rank, world_size=world_size)
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py", line 500, in init_process_group
    store, rank, world_size = next(rendezvous_iterator)
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/rendezvous.py", line 190, in _env_rendezvous_handler
    store = TCPStore(master_addr, master_port, world_size, start_daemon, timeout)
RuntimeError: Address already in use

Killing subprocess 3457985
Killing subprocess 3457986
Traceback (most recent call last):
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/home/wfang/anaconda3/envs/pytorch-env/lib/python3.9/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/wfang/anaconda3/envs/pytorch-env/bin/python', '-u', 'w1.py', '--local_rank=1']' returned non-zero exit status 1.

However, I can still run the codes in vision/references/classification at main · pytorch/vision · GitHub 1 with single machine and two GPUs.