Resuming Language Model Pre-training spends too much time skipping data
# 🚀 Feature request

Speed-up data skipping process when resuming from a checkpoint

## Motivation

Hi, currently resuming a language model pre-training from a checkpoint spends too much time skipping data.
<img width="1641" alt="Screenshot 2022-04-13 at 1 51 58 PM" src="https://user-images.githubusercontent.com/14203368/163404033-1232510a-efe7-4d9b-b4b0-699d49be234e.png">

While we can use `--ignore_data_skip` , the model would be trained on the already seen data. This would be problematic when we are training on huge corpus where I want the model to be trained on data not encountered to train till now.

If anyone has found a simpler solution, please let me know.

Hello! Could you please provide information on your environment, on the script used, etc? Thanks.<|||||>Hi, Thanks for the reply.
Please find the details

- `transformers` version: 4.18.0
- Platform: Red Hat Enterprise Linux release 8.5 (Ootpa)
- Python version: 3.8.11
- PyTorch version (GPU?):  1.9.1 (A100)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:  No. I am currently training on a single GPU

Please find the script used to train the model. I am using the language model training script from [here](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_mlm.py). I am training on a dataset containing `117 M` sentences

```bash
TRANSFORMERS_CACHE=/tmp/ PYTORCH_TRANSFORMERS_CACHE=/tmp/ PYTHONIOENCODING=utf-8 python src/lm/run_mlm.py \
    --model_type bert \
    --config_overrides="max_position_embeddings=512" \
    --remove_unused_columns False \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 8 \
    --train_file train.txt \
    --validation_file dev.txt \
    --line_by_line \
    --do_train \
    --do_eval \
    --preprocessing_num_workers 64 \
    --pad_to_max_length \
    --evaluation_strategy steps \
    --num_train_epochs 1 \
    --output_dir ./models/bert \
    --report_to tensorboard \
    --cache_dir /tmp/ \
    --logging_steps 10000 \
    --save_steps 10000 \
    --save_total_limit 2 \
    --tokenizer_name ./models/bert_tokenizer
```<|||||>cc @sgugger <|||||>I don't have a simpler solution.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.