Solved by Kushaj in post #3 
              
              
                This will not work. Both these states carry different meanings specific to the optimizer. Also, these values are largely dependent on the training process. 

No. This information is stored in the weights of the model, as weight.grad.