Solved by Wenhao-Yang in post #4 
              
              
                Thanks for your advice!  
I tried destroy_process_group() but it didn’t work. 
In the end, I fixed it by deleting the following Potential conflict line: 
torch.multiprocessing.set_sharing_strategy('file_system') 
It seems that this line is compatible with 
torch.distributed.init_process_group(backen…