Are you using system NCCL?

How to figure that out?
I changed to “gloo” and now I am getting -
AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [device(type='cuda', index=0)], output_device None, and module parameters {device(type='cuda', index=0), device(type='cpu')}.