I installed the pytorch-nightly using the following command
conda install pytorch torchvision cudatoolkit=10.2 -c pytorch-nightly

The NCCL version is 2.7.6 now.
The training is working fine so far. Thanks for the help @ptrblck.
However, I want to know what was the issue that was triggering the NCCL deadlock.