grads = {n:p.grad.cpu() for n, p in model.named_parameters()}

gives you the grads of model's parameters. You can now store them away, either directly on disk (torch.save or, if you feel fancy, hdf5) or keep a list of them (when moving to cpu probably is a good idea, so I threw that in above) or so.
Best regards
Thomas