[s2s trainer] tests to use distributed on multi-gpu machine
This PR:

* [x] abstracts the async forking io into local `utils.py`
* [x] deploys distributed training with special async io forking for `examples/seq2seq/test_finetune_trainer.py`

So now this works (2 gpus):
```
CUDA_VISIBLE_DEVICES="0,1" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py
```
and this still works (1 gpu)
```
CUDA_VISIBLE_DEVICES="0" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py
```

Fixes: #7833
Fixes: #7982

@sshleifer  
@sshleifer, the slow test isn't working for me prior to this PR with 0 or 1 gpu:

```
CUDA_VISIBLE_DEVICES="" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow   
[...]

self = <seq2seq.test_finetune_trainer.TestFinetuneTrainer testMethod=test_finetune_trainer_slow>

    @slow
    def test_finetune_trainer_slow(self):
        # There is a missing call to __init__process_group somewhere
>       output_dir = self.run_trainer(eval_steps=2, max_len="128", model_name=MARIAN_MODEL, num_train_epochs=3)

examples/seq2seq/test_finetune_trainer.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
examples/seq2seq/test_finetune_trainer.py:113: in run_trainer
    main()
examples/seq2seq/finetune_trainer.py:199: in main
    model = AutoModelForSeq2SeqLM.from_pretrained(
src/transformers/modeling_auto.py:1118: in from_pretrained
    return MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING[type(config)].from_pretrained(
src/transformers/modeling_utils.py:947: in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
src/transformers/modeling_bart.py:964: in __init__
    base_model = BartModel(config)
src/transformers/modeling_bart.py:843: in __init__
    self.encoder = BartEncoder(config, self.shared)
src/transformers/modeling_bart.py:315: in __init__
    self.embed_positions = SinusoidalPositionalEmbedding(
src/transformers/modeling_bart.py:1331: in __init__
    self.weight = self._init_weight(self.weight)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

out = Parameter containing:
tensor([[ 2.2244e+00, -1.2380e+00, -3.5307e-01,  ..., -1.0924e+00,
         -1.3130e+00,  1.7737...    [-3.8906e-01,  9.2203e-01,  1.7887e-01,  ..., -1.7493e-01,
         -1.6993e+00,  2.0896e-01]], requires_grad=True)

    @staticmethod
    def _init_weight(out: nn.Parameter):
        """Identical to the XLM create_sinusoidal_embeddings except features are not interleaved.
        The cos features are in the 2nd half of the vector. [dim // 2:]
        """
        n_pos, dim = out.shape
        position_enc = np.array(
            [[pos / np.power(10000, 2 * (j // 2) / dim) for j in range(dim)] for pos in range(n_pos)]
        )
>       out[:, 0 : dim // 2] = torch.FloatTensor(np.sin(position_enc[:, 0::2]))  # This line breaks for odd n_pos
E       RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation
```
**edit**: After messing around with my conda env due to ever-breaking tf, this went away, but a new thing came instead: https://github.com/huggingface/transformers/issues/7982
<|||||>interesting, I can't reproduce that. What's your `transformers-cli env`? Does it fail after the change?<|||||>No, it fails on master.

```
- `transformers` version: 3.4.0
- Platform: Linux-4.15.0-118-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201020 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```<|||||>I'd open an issue "sinusoidal positional embedding broken on torch 1.8".
Reasoing:  [these](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_bart.py#L606) fast tests pass CI and I can't replicate on on torch 1.5.

Is this ready to merge otherwise?<|||||>Need to sort out this first: https://github.com/huggingface/transformers/issues/7982

It's mostly ready otherwise, but the slow test will fail as it has nothing to do with this PR. 

**edit** resolved in this PR.<|||||>Out of curiosity, how did you resolve the bleu issue?<|||||>Great work, btw! This is awesome. Now we can tell people to run these tests before they break things :) 
Apparently there is multi-gpu ci running src/ tests at some frequency FYI, I think through gh actions.<|||||>I replied in the other issue: I used more iterations - 6 was enough for 1 gpu, 10 for 2, so I went with 10.

I think if someone tries it on more than 2 gpus it might need even more iterations - could probably codify this with a factor of n_gpus.
<|||||>Documenting it now https://github.com/huggingface/transformers/pull/7993
If you think anything needs to be added please let me know.
It will get better over time.<|||||>> Apparently there is multi-gpu ci running src/ tests at some frequency FYI, I think through gh actions.

Once a day yes:
https://github.com/huggingface/transformers/blob/master/.github/workflows/self-scheduled.yml#L74<|||||>Later I want to add the non-interactive IO pipe options - in case it hangs for someone - by default it could be non-interactive - always works, and only make it interactive for debug purposes.