My point is that your target new position does not depend on its previous position and neither on your previous action. If it is the case, you are not trying to solve a MDP, but something different, and actor critics may work but is not the optimal solution.