Thanks for the debugging.
As long as you recreate the optimizer, it should work.
You could check the optimizer.state_dict()['state'] for the running estimates. If they are not empty, something is wrong.
Otherwise, I agree that the result is looking unexpected, but the data might be the next thing to check.