If I call detach() on a variable after calling backward(retain_variables=True) on a variable further along the graph, will that free the buffers of the graph that led to the detach()'d variable?
Example: net1 and net2 are modules, x is an input variable. I want to retain the state of net2 for processing later down the line, but I just want to accumulate gradients in net1 for the time being, no need to retain the graph.
out1 = net1(x)
out2 = net2(out1)
loss = some_loss_function(out2)
loss.backward(retain_variables=True)
out1.detach_()
Will this clear the buffers of net1?
Thanks,
Andy