Tensor holds a reference to its .grad. When we set create_graph=True, we create the backward graph for .grad, so that we can perform double backward on it. In most cases this .grad will be a function of the original input, which means that there will be a chain of backward nodes leading back to the grad_accumulator of the input. Finally this grad_accumulator has an owning reference to its Tensor.
The cycle is: Tensor → .grad → .grad_fn → … → grad_accumulator → .variable → Tensor
When you call zero_grad, you don’t actually break the reference cycle because you are just setting the tensor’s values to zero, the tensor would still hold a reference to .grad, which still holds a reference to its gradfn.
To break the cycle you’d need to do x.grad=None.