Firstly,Thanks for the clear explanations!
For summary(if wrong,please do correct my statement):
If the object is referenced by more than just one ,after backward ,the only reference would be destroyed is computational graph one,the others that reference this object won’t be freed,If the object is only referenced by computational graph ,after backward,this reference would be freed and not be able to access it from  the computational graph.
And one more question is I would like to know what would be freed after back prop :
a = torch.tensor([2.0,3.0],requires_grad = True).view(2,1)
l = nn.Linear(1,3)
y  = l(a)
z = y*5 
o = torch.sum(z)
o.backward() 

During backward,the intermediate variables would be deleted,The word “intermediate” got me thinking that : Does it mean that the all the variables and functions related to operations “between” a and o would be freed or “include” a and o would be freed?
Thanks in advance!