[Deepspeed Inference] HF Integration
This PR is working on an integration of [Deepspeed Inference](https://www.deepspeed.ai/tutorials/inference-tutorial/) which implements Tensor Parallelism. This is different from Deepspeed ZeRO inference.

This is a very early draft.

To try:

```
cd transformers
export BS=16; rm -r output_dir; PYTHONPATH=src USE_TF=0  \
deepspeed --num_gpus=2 examples/pytorch/translation/run_translation.py \
--model_name_or_path t5-small --output_dir output_dir --adam_eps 1e-06 \
--evaluation_strategy=steps --do_eval --label_smoothing 0.1 --learning_rate \
3e-5 --logging_first_step --logging_steps 500 --max_source_length 128 \
--max_target_length 128 --overwrite_output_dir --per_device_eval_batch_size $BS \
--predict_with_generate --sortish_sampler --source_lang en --target_lang ro \
--dataset_name wmt16 --dataset_config ro-en --source_prefix \
'translate English to Romanian: ' --val_max_target_length 128 --warmup_steps \
50 --max_eval_samples 50 --deepspeed_inference --skip_memory_metrics 0
```

and it currently hangs with `--num_gpus > 1`. One gpu finishes processing and the other is stuck in preparing inputs. So need to figure out the synchronization of the gpus.
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_14426). All of your documentation changes will be reflected on that endpoint.