“Image” here refer to as the docker image. Two machines share the same docker image, while only one of them works fine.

That’s really weird, so the software setup should be exactly the same, and you also ran exactly the same code. Have you tried any different programs that can run on multiple GPUs? For example, another DDP training use case or just running some collective communication APIs like torch.distributed.all_reduce?
I am afraid that I may not be able to reproduce your issue on my own setup.