Hi jomavera,
thank you so much for the super fast reply, even for necroing this post 
I will try this right now and follow up with the results. In particular this line of code, i think the i iteration is for each sample in batch, not for each action. But i will confirm it right now



 jomavera:

for i in range(policy_output.size()[0]):


Ah yes, the i is for sample iteration and the previous error i mentioned is this:
   grad_inverter = T.where(T.gt(grads, zeros_grad), T.mul(grads,pdiff_max), T.mul(grads,pdiff_min))
RuntimeError: The size of tensor a (99) must match the size of tensor b (8) at non-singleton dimension 1

when i print
print(policy_output.size()) = torch.Size([64, 1])