In each minibatch gradient descent, the computation graph is created once, then we backprop the loss to update the model weight. Is that right?
Also I would like to know what kind of intermediate results are freed so that we can not backward()  the final loss anymore, unless we use retain_graph=True. AFAIK, the computation graph is composed of variables and math operations. Since the variables remains, does it imply that the freed info is a series of math operations that lead to the final loss?