That sounds like a good idea.
It would just reset the internal states for some optimizers or do you have anything else in mind?
Would you like to create this feature request on GitHub 135 and explain your use case a bit?
Also, would you be interested on implementing this feature in case the proposal gets some positive feedback?