Distributed multi-node support for CPU cluster
# 🚀 Feature request

## Motivation
Current distributed run supports only multi-GPU/TPU run. This feature requests adding support for distributed CPU runs using MPI/GLOO or recently added custom backend e.g. intel oneCCL backend using [torch-ccl](https://github.com/intel/torch-ccl) plugin.

## Your contribution
An example use for Intel oneCCL backend (using Intel torch-ccl) can be found at https://github.com/ddkalamk/transformers/blob/pcl-v4.0.0/examples/question-answering/run_squad.py#L746

Assumption is we launch the application using MPI (similar to horovod) and based on environment variable initialize distributed backend.

Here is another and more comprehensive use case from Facebook DLRM workload:
https://github.com/facebookresearch/dlrm/blob/master/extend_distributed.py#L59


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.