Sure, I’ll clarify: the question is how to
“start the autograd backprop from accumulated gradient instead of from loss or activation”
That is, we want dout/da , but we only have dout/dp (we do not have out), so we can’t compute it straightforward.
Mathematically:
dout/da = dout/dp * dp/da
all we need is dp/da and we are done.
note1: a is arbitrary, can be an activation or another parameter.
note2: we can create the computational graph only from a->p.