[seq2seq distributed] child process stuck on error

Filing an issue for myself to help move along https://github.com/huggingface/transformers/pull/7281
```
pytest -sv examples/seq2seq/test_seq2seq_examples_multi_gpu.py
```
gets stuck if one of the child subprocesses throws an error.

Remove `skip_output_dir_check=True,` to reproduce the problem.

It could be related to realtime async read/write pipes getting into a deadlock - see the XXX warning in that test, need to first try whether the problem goes away if I switch to non-real time pipes.

I will get back to it later.


OK, this is the problem with PL if anything, as the command:
```
PYTHONPATH="src" /home/stas/anaconda3/envs/main-38/bin/python /mnt/nvme1/code/huggingface/transformers-distr-train-test/examples/seq2seq/distillation.py --supervise_forward --normalize_hidden --label_smoothing=0.0 --eval_beams=1 --val_metric=loss --save_top_k=1 --adafactor --early_stopping_patience=-1 --logger_name=default --length_penalty=0.5 --cache_dir= --task=summarization --num_workers=2 --alpha_hid=0 --freeze_embeds --sortish_sampler --student_decoder_layers=1 --val_check_interval=0.5 --output_dir=/tmp/tmpqajqhzwo --no_teacher --fp16_opt_level=O1 --gpus=2 --max_grad_norm=1.0 --do_train --do_predict --accumulate_grad_batches=1 --seed=42 --model_name_or_path=sshleifer/tinier_bart --config_name= --tokenizer_name=facebook/bart-large --learning_rate=0.3 --lr_scheduler=linear --weight_decay=0.0 --adam_epsilon=1e-08 --warmup_steps=0 --max_epochs=2 --train_batch_size=1 --eval_batch_size=2 --max_source_length=12 --max_target_length=12 --val_max_target_length=12 --test_max_target_length=12 --n_train=-1 --n_val=-1 --n_test=-1 --student_encoder_layers=1 --freeze_encoder --data_dir=/tmp/tmpo_9t6k7e --alpha_mlm=0.2 --alpha_ce=0.8 --teacher=sshleifer/bart-tiny-random
```
hangs on its own, so this is definitely not an issue of the newly added distributed support for `pytest`. 

I think that when one of the workers fails (e.g. asserts on output directory already exists) and the coordinating process gets stuck waiting for the worker to send something back.