Use `accelerate` with transformers==4.26.1
### Feature request

How can we use `accelerate` features with an older version of `transformers`? I'm asking this because I've to use `adapter-transformers` and that is based on `transformers v4.26.1`


### Motivation

I used it latest `transformers` version and found it really helpful for parallelising the training 

### Your contribution

It's more of an optimization thing for an older version, I'm not sure how helpful will it be for other people or if any PR will be required for this.

Tagging @muellerzr here as he has worked on this perviously. Thank you!
You'd need to recreate everything that has been performed throughout this integration, which is quite a lot. Distributed training still works in the Trainer natively, we just replaced its guts with Accelerate. The only real main difference is the DataLoader setup is a bit different in how it handles the batches. However in terms of usage nothing truly has changed. So they should update their version to v4.30.1 if you'd like this.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.