Solved by ptrblck in post #2 
              
              
                The PyTorch binaries ship with a statically linked NCCL using the NCCL submodule. The current CUDA11.3 nightly binary uses NCCL 2.10.3 already, so you could use it. 
On the other hand, if you want to use a specific NCCL version, which isn’t shipped in a binary release, you could build from source an…