The state_dict doesn’t contain the .grad attributes of the parameters, but you could try to store a custom dict with them and reassign these gradients after loading the state_dict to all needed parameters.
EDIT: state_dict hooks might also be an valuable workflow you could take a look into.