I left cuda lines just to illustrate that I have no leftover tensors at that point, and re-run acts as if process was restarted (except explicitly stored stuff of course)
Regarding hooks, be sure to .detach() recorded tensors, and note that their installers return “handles” to invoke .remove() on. I don’t use them though, so not sure if they have any additional inherent problems…