Solved by pritamdamania87 in post #2 
              
              
                You can launch 4 processes (1 per GPU) and initialize a process group of world_size 4. You can use this process group to exchange data between A and B. 
Then for each model, create a new subprocess group using Distributed communication package - torch.distributed — PyTorch 1.9.0 documentation on GPU…