Edit: clarification:
The question is how to
“ start  the autograd backprop from  accumulated gradient  instead of from loss or activation”
That is, we want  dout/da  , but we only have  dout/dp  (we do not have  out ), so we can’t compute it straightforward.
Mathematically:
dout/da = dout/dp * dp/da
all we need is  dp/da  and we are done.
note1:  a  is arbitrary, can be an activation or another parameter.
note2: we can create the computational graph only from  a->p .

Original post:
I know of two main ways to do backprop in pytorch:

from loss: loss.backward()

using gradients and activation with same size (or broadcast-able?) torch.autograd.backward(x, g)


Now, lets say we have some  p = torch.nn.Parameter() for which some gradient (p.grad) was already accumulated, and we want to continue the back-propagation from it.
Lets say we can construct the entire computational graph until p, and now we have it as some activation: a=f(input)
Note that mathematically its very simple chain rule, just not sure how to do it in pytorch. (at least from python)
Any suggestions?
Here is some pseudo code for the problem:
input=torch.tensor(...,requires_grad=True)
a=f(input)
p.grad = some_value_we_want # dout/dp
p.data = another_value_we_want 

How do we calculate dout/da in pytorch?
assuming
out = p ? f(input) where ? is some unknown function.
(shape of a is not the same shape of p.grad)…