save_for_backward saves the current tensor plus the “inplace version counter” separately and then at backwards compares the version counter of the tensor whether it still matches the saved version (throwing the infamous a tensor needed for backwards has been modified …).
The thing to probably avoid is some circular dependency at the C++ shared pointer level as that will produce memory leaks, but I’m not sure why it should do that.