That would certainly work, but I still have to modify the code in all the RNN models to save the gradients, which is not what I want. The recording and the model itself should be separate. It’s also not what the documentation implies. Taken directly from there:
def register_hook(self, hook):
        r"""Registers a backward hook.

        The hook will be called every time a gradient with respect to the
        Tensor is computed. The hook should have the following signature::

            hook(grad) -> Tensor or None
...

Which implies that it should be called at each time step, since we compute a gradient w.r.t. the parameters for each of them. Yet when I do this, I only get one gradient.
Maybe someone want’s the gradient for the whole sequence, but that can be obtained using the register_backward_hook method in the Module class.