Are you trying to train your model for only 1 epoch because you have so much data and it’ll take too long to do more, or are you possibly trying to do 1 epoch because your machine won’t allow it to finish and everything shuts off, so you’d like to save intermediate progress? (Epoch = single pass through your entire dataset) Just asking out of curiousity, no worries if there’s no reason.
As for for your question, I’d do one of the following:


Drop shuffle=True and as you train keep track of an id (either the step number, which will represent what batch you are on, or just the raw id of current sample you’re on). If you’re using a HuggingFace Trainer instance for your model training, you can use callbacks to do this (add a on_step_end or on_step_begin callback to write out current step # to a file, can be found here in the docs). When continuing training, You can slice examples starting from the id you left on, and ending with the last id of the dataset, then append all the samples you’ve already trained with at the end of this slice (essentially shifting the samples you trained with, but putting them at the end). If you don’t care about re-using the samples at the end, you can just use PyTorch’s Subset dataset class.


Keep shuffle=True but have a small function call when you fetch a sample that writes-out the id that’s getting fetched/processed. When continuing training, do a similar process as above (option 1) but rather than working with a single slice from shuffle=False you can slice out a subset of your dataset using the ids you’ve saved