resume train
### System Info

- `transformers` version: 4.28.0.dev0
- Platform: Linux-4.15.0-175-generic-x86_64-with-glibc2.31
- Python version: 3.10.10
- Huggingface_hub version: 0.13.3
- Safetensors version: not installed
- PyTorch version (GPU?): 1.12.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Currently, I am using 8 GPUs to train GPT. I used the `Trainer` API provided by huggingface for training. Due to the large amount of data, it is expected to train only one epoch. When I was halfway through my training, I stopped training and increased the number of GPU to 24. When I resumed training, there was no change in the number of steps trained. However, due to the increase in the number of GPUs, the global batch size will also increase, so theoretically, the overall training step should change.

### Expected behavior

Is this a displayed bug? In other words, the steps required to complete an epoch during training will be less than the displayed number of steps.
If not, how should I skip data that has already been trained?
Resuming training with a different setup than the one that begun it is at your own peril and is definitely not recommended or officially supported.<|||||>Ok, thank you for your reply. I will attempt to manually skip these already trained data.