You’re right, but aren’t they freed after autograd finishes its job in backward? If I am not mistaken autograd uses these intermediate derivatives to calculate the gradient in leaves. So I think we can access them somewhere in c-sources.
Yeah maybe I have to look at autograd to see if I can find something there.