Request: Ignore Dataset transforms when iterating to the most recent checkpoint when resuming training
# 🚀 Feature request

It'd be great if, when resuming training from a checkpoint and using a Dataset with a format/transform function applied, the dataset's format/transform function could be ignored while iterating up to the last checkpoint step.

@lhoestq @sgugger 

## Motivation

I doubt it's much of an issue most of the time, but I've started playing with `dataset.set_transform()` for doing some heavy preprocessing, and just iterating through samples to the current checkpoint step can take a ridiculously long time compared to a dataset without a transform applied. And I don't think there's any case where the transformed sample would be used, right?

See [this conversation in the forum](https://discuss.huggingface.co/t/understanding-set-transform/3740/6?u=jncasey) for more backstory and my rudimentary thoughts on how I'd accomplish it.

## Your contribution

I'm hesitant to try updating any of the trainer code myself since it's so complicated, and needs to cover so many edge cases I'm not familiar with. 

This is already there :-) Just pass along `--ignore_data_skip` in your script or `ignore_data_skip=True` in your `TrainingArguments`.<|||||>Wow, that was fast! :) 

That loads the model from the checkpoint and advances the dataset to the next sample that would have been trained in the original run?

From my reading of the code I assumed that it reloaded the model and started the training over at the first sample of the dataset.
<|||||>Ah sorry I misunderstood your feature request. Indeed it starts from the first sample instead of iterating. What you ask is a bit more complicated and will require a minimum version of datasets. It's possible but will take some time.<|||||>Sorry for the confusion from my writing! And thanks, as always, for your work on this amazing project.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.