@alexis-jacq
I think what was described above is
Dynamics.requires_grad = False
P.requires_grad = True

S0 = leaf Variable
A0 = P(S0)
S1  = Dynamics(S0, A0)
C1 = cost(S1)
A1 = P(S1)
S2 = Dynamics(S1, A1)
C2 = cost(C2)

C1.backward()
P_optimizer.zero_grad()
Save grad of C1
C2.backward()
Add grad of C1 with grad of C2

Update parameter of P

This may not be the right way, I guess the topic means during backward(), the P.grad only stored in P(S0), and when executing P(S1) the gradients just flows through
i.e. for P(S1): requires_grad = False and P(S0): requires_grad = True