Are you storing and loading the model or optimizer in some way.
There were some threads on this topic in the past few weeks and so far most of the time we’ve found some code bugs in the training procedure.
I’m really interested in a working example.