I assumed in the code that the action is only one component/scalar. The invert_gradient function should be called for each action component independently so the sum I don’t think is appropiate. So if we “compute” an inverted gradient for each action component and each at batch sample and sum it to params_grads[name]  then I think the gradient should be the mean w.r.t. components and batch
param.grad.data = params_grads[name]/(batch_size*action_size)