Step restarts from step 0 when reload from an existing checkpoint?
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hi, everyone

I am totally new to Transformers, it's really a good solution. :-)

Question:

When I reload my program from a break point (existing checkpoint, say checkpoint-30000),
what I expect is that, from Tensorboard, I can see the program restarting from 30001.

However, it restarts from step 0, although parameters are up to date.

Does any config point to this problem?

Or any easy solutions?

Thanks.
