What’s the easiest way to reset an optimizer stats, such as Adam’s moving averages, while keeping the same weights?
To make an example, suppose I have a model and I have pretrained it on a dataset using Adam. Now, I want to reset Adam’s stats and train the model on another dataset, while keeping the same parameters to be optimized. What’s the best way to reset the optimizers state, except for param_groups?