Thanks for the answer, I considered also that way, however I was confused by param_groups and defaults and decided to tweak directly the state. All in all, I would consider to add a method reset, to be able to reuse the same optimizer. Would it make sense?