Does save_for_backward() work for operations like RNN cells where the same operation instance is forwarded multiple times before backward is called, or does anything special need to be done for this use case?


Yes. It works fine.
This is because RNN cells are of type nn.Module, but save_for_backward is really implemented inside autograd.Function. All autograd Functions inside a graph only have single instantiations. If you have K time steps, you will have K instantiations of the relevant functions. nn.Module are abstractions on top of autograd Functions that make this seamless.

if you are referring to nn.RNN there are limitations documented by the API itself. But you can create your own RNN as you wish.