IIUC, what you have in mind is the reverse structure of this tutorial 10. In the tutorial, there are multiple observers sending inputs to the same agent, while in your case, you would like to have the server sending inputs to different clients and run forward on the clients?
The problem with the server-client structure is that, if forward is run on client, the autograd graph and activations will also be on client, meaning that the server cannot merge the output and run the backward locally.
One possible alternative is that, instead of sending forward output from client to server, you can let the client finish forward-backward and then send the model param gradients to server. Then the server collects gradients from all clients and sum them, use the summed grads to update parameters, and then broadcast the updated model to all clients.
Another alternative is to let the client finish forward-backward-optimizer, and then send the model params to the server. Then the server calculates the weighted average of all params and broadcast them back to the clients.