Thank you Alban!
It’s not the parameters here, btw. Here is what happens as far as I can tell:

Obviously, xs holds a reference to xs.grad if that is populated, as it is a member.
But via the graph built with backward(..., create_graph=True) you have that xs.grad via its grad_fn and some next_functions has an indirect reference to xs.

This reference cycle isn’t just in Python but rather in C++, so while the Python bits will go out of scope and can be garbage collected eventually, the C++ doesn’t have garbage collection and so the C++ side of the tensors will remain alive.
If you set xs.grad to  None, it’ll remove the C++ link and so the C++ bits will be deallocated as well.
Best regards
Thomas