Saving and loading the model doesn’t restore gradients nor tensors assigned as attributes. It will save and restore parameters (nn.Parameter) and buffers. As your gradient is not a parameter, you could use register_buffer or so.
Whether or not saving and restoring the gradient is a good idea in general is another topic, but as long as it’s code for yourself, you’re the arbiter of this. I would probably advise against doing so for things you want to put into a library or somesuch.
Best regards
Thomas