Hi,
I’m trying to train two models A and B on 4 GPUs, each being trained on 2 GPUs (and thus DDP is needed). The two models are independent, but they need to exchange some information during training (not gradients), hence I would like to execute a single command torch.distributed.launch so that the workers can communicate to each other. What is the correct way to do it?
Thanks in advance!