When you are using an optimizer on some parameters, only these parameter will be affected by the gradient step. So, if you did something like
policy_optim = nn.Optimizer(policy.parameters(), lr)

above in your code, and then
policy_loss = -Q(s, policy(s))
policy_loss.backward()
policy_optim.step()

… then, only the parameter of the policy will be affected, with the gradient of your loss wrt these parameters. It’s that simple!