xla_spawn.py:  xm.get_ordinal() got an unexpected keyword argument 'local'
## Environment info

- `transformers` version: 4.7.0.dev0
- Platform: Linux-5.4.109+-x86_64-with-Ubuntu-18.04-bionic (working on Colab with TPU)
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu101 (False)
- Tensorflow version (GPU?): 2.5.0 (False)
- Using GPU in script?: No, using TPU
- Using distributed or parallel set-up in script?: number_of_cores = 8

### Who can help
<@sgugger, @patil-suraj -->

## Information

Model I am using: Bert

The problem arises when using:
* [ ] the official example scripts:
```ruby
python /transformers/examples/pytorch/xla_spawn.py --num_cores=8 \
           /transformers/examples/pytorch/language-modeling/run_mlm.py (--run_mlm.py args)
```

The tasks I am working on is:
* Pretraining BERT with TPU 

## To reproduce

Steps to reproduce the behavior:

1. install necessary packages:

```ruby
pip install git+https://github.com/huggingface/transformers
cd /content/transformers/examples/pytorch/language-modeling
pip install -r requirements.txt
pip install cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.8.1-cp37-cp37m-linux_x86_64.whl
```

2. run xla_spwan with minimal args passed to run_mlm: specify a small .txt TRAIN_FILE and an OUTPUT_DIR:

```ruby
python xla_spawn.py \
        --num_cores=8 \
        language-modeling/run_mlm.py \
            --train_file $TRAIN_FILE \
            --model_name_or_path bert-base-uncased \
            --output_dir $OUTPUT_DIR \
            --overwrite_output_dir True \
            --do_train True \
```

I get this error (for different TPU cores):

```ruby
Exception in device=TPU:0: get_ordinal() got an unexpected keyword argument 'local'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 329, in _mp_start_fn
    _start_fn(index, pf_cfg, fn, args)
  File "/usr/local/lib/python3.7/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 323, in _start_fn
    fn(gindex, *args)
  File "/content/drive/My Drive/Thesis/transformers/examples/pytorch/language-modeling/run_mlm.py", line 493, in _mp_fn
    main()
  File "/content/drive/My Drive/Thesis/transformers/examples/pytorch/language-modeling/run_mlm.py", line 451, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer.py", line 1193, in train
    self.state.is_local_process_zero = self.is_local_process_zero()
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer.py", line 1784, in is_local_process_zero
    return self.args.local_process_index == 0
  File "/usr/local/lib/python3.7/dist-packages/transformers/file_utils.py", line 1605, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/transformers/training_args.py", line 864, in local_process_index
    return xm.get_ordinal(local=True)
TypeError: get_ordinal() got an unexpected keyword argument 'local'
```

## Expected behavior

The training should run without errors. I achieved this by simply replacing line 864 of /transformers/training_args.py:
```ruby
            return xm.get_ordinal(local=True)
```
with:
```ruby
            return xm.get_local_ordinal()
```
Following torch docs at:
            https://pytorch.org/xla/release/1.5/_modules/torch_xla/core/xla_model.html

If this is the correct syntax (and this behaviour is not due to something wrong in my environment), this easy fix should be enough. My model trained correctly.

Thanks for the catch! Since you have the proper fix indeed, would like to make a PR with it?<|||||>Done, thanks!<|||||>Closed by #11922