Hi,
In my opinion, you should make sure to get samples in the same order, for that you can set she dataset shuffle=False or use a sampler to get samples, and then skip the first batch_id of the dataset, so that we can continue the last training phase state.
But why do you want to save the last training batch_id or batch_data, if you set dataset shuffle=True, you just need to complete the remaining epochs you don’t complete in your last training.