Hi, I have the same problem, but I have found the solution. The pytorch have supported backend="gloo" while the support for nccl has not been completed. I have tested it in pytorch nightly. If the stable version still go wrong, you can update to it.