Hi,
I want to train several replicas of the same network. After each replica has completed, the weights will be exchanged. Although tensors are exchanged during runtime, after each replica there will be tensor exchanges. Can multiprocessing handle this?