so back to my question, how is the gradient wrt action specified?  Q(s,a) has both state and action as variable, and the gradient can be taken wrt to state if it’s not specified. This is what’s confusing me.