sorry, I should say I am trying to find the actor gradient. I am referencing this implementation 87 and he does
policy_loss = -self.critic([to_tensor(state_batch), self.actor(to_tensor(state_batch))].mean()
which is just simply defining the loss = -mean(critic(state,actor(state)))
My belief is that the autograd will find the gradient with respect to action, which is actor_grad = Q_grad * mu_grad given the chain rule property.