Oh! PyTorch internals time!
So what happens is that save_for_backward stores the tensors in SavedVariable 61s. This is what does the sanity checks for the backward (one of the variables needed for gradient computation has been modified by an inplace operation being the perhaps most (in)famous).
Now variables that aren’t input or output of your function (and don’t share storage…) are not visible to the outside world and thus cannot be changed in ways that would trigger those sanity checks.
Now you can safely store other tensors, but SavedVariable won’t work for non-tensors.
Best regards
Thomas