Hi, I have met a situation which was little different with @leo-mao. I want to train my model using one machine (node) which has multi-GPUs with torch.distributed. Since each time I just used 2 GPUs, I want to run several models at the same time. The problem is when I have started one model running with torch.distributed, others will get an error info " RuntimeError: Address already in use ". I set the initial way as @teng-li says. Maybe I ignore something like port setting ? I am confused about it. I will be very appreciate if someone can give me some suggestions.