Have one as master server. I don’t understand the context. In case of CPU you have torch.multiprocessing using which you can create new threads sharing the same data (i.e. your model) and execute those threads on different cores.
In case of GPU you will have to load the model onto each GPU memory as far as I understand.