@alexis-jacq thank you for pointing that example.
let me compare actor_critic.py and reinforce.py
final_nodes = [value_loss] + list(map(lambda p: p.action, saved_actions))
gradients = [torch.ones(1)] + [None] * len(saved_actions)
autograd.backward(final_nodes, gradients)

autograd.backward(policy.saved_actions, [None for _ in policy.saved_actions])

I found the reason why value_loss requires 1 while saved_actions do [None, ...] as follows;

these value are initial backprop derivatives dL/dy in a chain rule: dL/dx = dL/dy * dy/dx

value_loss's initial gradient is 1 as a usual regression loss: (y=L)

and reinforce.py  does not have such a loss. it use mean/var simply instead of trainable normalizer (i.e., value net) in actor_critic.py




saved_actions have no backprop derivatives because they are detached as seen in my snippet.

is it right?