Did you use some kind of lr_scheduler?
Your saveing code looks alright so far.
I’ve created a small gist 148 a while ago, since another user reported a similar issue.
Could you compare your code to mine and check, if there are any logical differences?