Another question if I may, turns out it produce error of mismatch dimension
pdiff_max 's dimension is n_actions
while grads is of n_states
is it maybe because n_actions > 1
and if n_actions > 1, do we just sum pdiff_max and pdiff_min ?
pdiff_max = torch.sum(torch.div(-action+1.0,2.0))
pdiff_min = torch.sum(torch.div(action+1.0,2.0))

because we are doing it for every action and add all.
Thank you once again