Kale-ab_Tessera:

Do I simply get the state info and use that to compare optimizers e.g. state['exp_avg'] (from Adam) and compare it to state['sum'] (from Adagrad)?


This will not work. Both these states carry different meanings specific to the optimizer. Also, these values are largely dependent on the training process.



 Kale-ab_Tessera:

Can I get gradient or activation information from the optimizer’s state dict?


No. This information is stored in the weights of the model, as weight.grad.