Sorry for bumping this but I have the same problem that @nadia described. I’ve been able to run the first example described in https://pytorch.org/tutorials/intermediate/dist_tuto.html 44 with the gloo and nccl backends, which makes me believe this might not be related to an NCCL deadlock, but things just freeze when I do model = torch.nn.parallel.DistributedDataParallel(model).