Yes it’s just like in federated learning. client A and client B will be training their own data for respective number of epochs E. Then they need to send the gradients to a master who will average the model and send it back to the clients. I do have some access to Linux system. I would appreciate if you could provide me an example to do this.
P.S. Can you tell how to make sure that each of the model has different input? currently all of the examples use a same dataloader to get the input samples and just randomly distribute the data as input. I need to have each of the model train separately on their own data.