Have you tried other backend types (Gloo, MPI), do they fail with the same error?
How do you initialize process group and construct DistributedDataParallel?
For debugging, we would first try a minimum DDP example like this one 85 and make sure it works correctly with the given environment. And then switch to more complex models.