understood, so the learnable parameters are the ones that dictate how the actuator movements are predicted. i’ll suggest a scheme with a notation different from yours, i hope it can be of use
current position/distance to player : x_now
actuator movement: y_now = net(x_now)
next position: x_next = x_now + y_now ( i am simplifying this as addition of actuator movement to position, you can of course have whatever physical function that dictates how action changes position)
loss : L(x_next,x_target) , e.g. (x_next - x_target)**2
due to x_next involving y_now which in turn involves net params, now you will get non zero gradients for the parameters