The first evident error in your code is:
msd = model.state_dict()
for name, param in model.named_parameters():
    msd[name].grad = conn.tensorget(f'{name}_grad')
model.load_state_dict(msd)

Since load_state_dict will not load the gradient, but only load registered nn.Parameter() and buffers of model, you will have to iterate parameters of your model directly and put gradients there like:
                ...
                # Assign gradients to the managed model and
                # perform optimization.
                if self.model is not None and self.optimizer is not None:
                    self.optimizer.zero_grad()
                    with t.no_grad():
                        for k, v in self.model.parameters():
                            v.grad = grad_dict[k].to(v.device)
                    self.optimizer.step()

Therefore, the “solution” you have discovered is basically performing a optimization step on your “PS” end, but your pushed gradients are not utilized.