You can (1) look at the tutorial of distributed parallel training (for the sake of different models on different device and your loss computation), and (2) imitate the structure of parameter server to keep the model possessing the same set of parameters.