SmoothPQ:

Thank you very much for the answer. Is nccl installed automatically when installing CUDA on Linux, or do I need to add something else?


On Linux, NCCL and torch.distributed will be enabled by default. On MacOs, with PyTorch 1.3.1+, you need to conda install libuv and pkg-config explicitly set USE_DISTRIBUTED=1 when compiling from source. For Windows, torch.distributed is not enabled yet.