Thanks @iffiX. This comment is the most relevant answer to my question. I will test  register_backward_hook function to manually control the gradient of my network(grads).
N = states.shape[0]

loss = T.log(prob_act[T.arange(N), actions]).dot(values) / N  # call it "loss"

grads = T.grad(loss, params)