[fairscale] rng states saving fails in an extended multi-gpu test
@sgugger, following up on the RNG states PR
```
RUN_SLOW=1 pytest tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_fully_sharded_ddp_fp16 -sv
```
fails on multi-gpu scheduled CI:

```
File "/__w/transformers/transformers/examples/pytorch/translation/run_translation.py", line 589, in <module>
  main()
File "/__w/transformers/transformers/examples/pytorch/translation/run_translation.py", line 522, in main
  train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/__w/transformers/transformers/src/transformers/trainer.py", line 1316, in train
  self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
File "/__w/transformers/transformers/src/transformers/trainer.py", line 1397, in _maybe_log_save_evaluate
  self._save_checkpoint(model, trial, metrics=metrics)
File "/__w/transformers/transformers/src/transformers/trainer.py", line 1534, in _save_checkpoint
stderr: Saving model checkpoint to /tmp/tmpb6tfhvqb/checkpoint-1
  torch.save(rng_states, os.path.join(output_dir, f"rng_state_{local_rank}.pth"))
File "/opt/conda/lib/python3.8/site-packages/torch/serialization.py", line 369, in save
  with _open_file_like(f, 'wb') as opened_file:
File "/opt/conda/lib/python3.8/site-packages/torch/serialization.py", line 230, in _open_file_like
  return _open_file(name_or_buffer, mode)
File "/opt/conda/lib/python3.8/site-packages/torch/serialization.py", line 211, in __init__
  super(_open_file, self).__init__(open(name, mode))
stderr: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpb6tfhvqb/checkpoint-1/rng_state_1.pth'
```

https://github.com/huggingface/transformers/runs/2514006341?check_suite_focus=true

Oddly enough it works fine for me if I try it locally.


Found the root: the "checkpoint-1" folder is only created by process 0 when calling "model.save_pretrained", so if process 1 is faster and arrives at `_save_checkpoint` first, it will not see this folder (there is almost no saving on process 1, so it goes through this method quickly).

Will send a fix shortly.