Adam and other optimizers has “parameters”. These parameters are crucial in order to guide backpropagation and gradients like momentum does in SGD.
If you don’t save optimizer’ state dict, these parameters are wrong once you re-load the model and make the backprop step to point in a wrong direction and goes far away from previous point.
Is it possible?