Hyperparameter search w/ RayTune BrokenPipeError: [Errno 32] Broken pipe
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: Google Colab
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7
- Using GPU in script?: Yes

### Who can help
- ray/raytune: @richardliaw, @amogkam
- trainer: @sgugger

### Information

Model I am using (Bert, XLNet ...): sshleifer/distilbart-cnn-12-6
Dataset: dummy XSUM (50 samples in train, 5 samples in val)

## To reproduce

I have tried `trainer.train` with the exact same parameters and it works just fine.

I am trying to do a hyperparameter search with the Seq2SeqTrainer and RayTune. For now I am just trying a dummy search with 2 different learning rates and 2 different gradient accumulation steps. Here is my code:

```
def hp_objective(metrics):
    loss = metrics.pop('eval_loss', None)
    _ = metrics.pop('epoch', None)
    _ = metrics.pop('eval_gen_len', None)

    return np.sum(list(metrics.values()))

def hp_space(trial):
    from ray import tune

    return {
    'learning_rate': tune.choice([1e-5, 1e-4]),
    'gradient_accumulation_steps': tune.choice([4, 8])
    }

def model_init():
            model = AutoModelForSeq2SeqLM.from_pretrained(
                model_args.model_name_or_path,
                config=config,
                cache_dir=model_args.cache_dir,
            )
            # use task specific params
            use_task_specific_params(model, data_args.task)

            # set num_beams for evaluation
            if data_args.eval_beams is None:
                data_args.eval_beams = model.config.num_beams

            # set decoder_start_token_id for MBart
            if model.config.decoder_start_token_id is None and isinstance(tokenizer, MBartTokenizer):
                assert (
                    data_args.tgt_lang is not None and data_args.src_lang is not None
                ), "mBart requires --tgt_lang and --src_lang"
                model.config.decoder_start_token_id = tokenizer.lang_code_to_id[data_args.tgt_lang]

            if model_args.freeze_embeds:
                freeze_embeds(model)
            if model_args.freeze_encoder:
                freeze_params(model.get_encoder())
                assert_all_frozen(model.get_encoder())

            return model

trainer = Seq2SeqTrainer(
            model_init=model_init,
            config=config,
            args=training_args,
            train_dataset=train_dataset,
            eval_dataset=eval_dataset,
            data_collator=Seq2SeqDataCollator(tokenizer, data_args, training_args.tpu_num_cores),
            compute_metrics=compute_metrics_fn,
            data_args=data_args)

logger.info("*** Hyperparameters Search ***")
start_time = time.time()

trainer.hyperparameter_search(
            direction = "maximize",
            compute_objective = hp_objective,
            hp_space = hp_space,
            backend = "ray",
            resources_per_trial = {'gpu': 1})
```

And I get the following error:

```
02/01/2021 16:26:31 - INFO - __main__ -   *** Hyperparameters Search ***
02/01/2021 16:26:31 - INFO - ray.tune.ray_trial_executor -   Initializing Ray automatically.For cluster usage or custom Ray initialization, call `ray.init(...)` before `tune.run`.
2021-02-01 16:26:33,796	INFO services.py:1173 -- View the Ray dashboard at http://127.0.0.1:8265
tcmalloc: large alloc 1236656128 bytes == 0x7f275ac1a000 @  0x7f2ab6d16615 0x591e47 0x4cc179 0x4cc2db 0x566a71 0x5a4cd1 0x5a4fb8 0x7f2a1e822c7c 0x7f2a1e829bfa 0x7f2a1e829fe2 0x7f2a1e82b34f 0x7f2a1e828a39 0x7f2a1e829afc 0x7f2a1e82b34f 0x7f2a1e828a39 0x7f2a1e829fe2 0x7f2a1e82b34f 0x7f2a1e828a39 0x7f2a1e82ad13 0x7f2a1e82b43c 0x7f2a1e828a39 0x7f2a1e829f45 0x7f2a1e82b7fa 0x7f2a1e828a39 0x7f2a1e82ad5e 0x7f2a1e82b43c 0x7f2a1e828a39 0x7f2a1e829f45 0x7f2a1e82b7fa 0x7f2a1e828a39 0x7f2a1e82ad5e
tcmalloc: large alloc 1545822208 bytes == 0x7f26fe9e4000 @  0x7f2ab6d16615 0x591e47 0x4cc179 0x4cc2db 0x566a71 0x5a4cd1 0x5a4fb8 0x7f2a1e822c7c 0x7f2a1e829bfa 0x7f2a1e829fe2 0x7f2a1e82b34f 0x7f2a1e828a39 0x7f2a1e829afc 0x7f2a1e82b34f 0x7f2a1e828a39 0x7f2a1e829fe2 0x7f2a1e82b34f 0x7f2a1e828a39 0x7f2a1e82ad13 0x7f2a1e82b43c 0x7f2a1e828a39 0x7f2a1e829f45 0x7f2a1e82b7fa 0x7f2a1e828a39 0x7f2a1e82ad5e 0x7f2a1e82b43c 0x7f2a1e828a39 0x7f2a1e829f45 0x7f2a1e82b7fa 0x7f2a1e828a39 0x7f2a1e82ad13
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 706, in send_packed_command
    sendall(self._sock, item)
  File "/usr/local/lib/python3.6/dist-packages/redis/_compat.py", line 9, in sendall
    return sock.sendall(*args, **kwargs)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/content/drive/My Drive/MAGMA: Summarization/transformers/examples/seq2seq/finetune_trainer.py", line 436, in <module>
    main()
  File "/content/drive/My Drive/MAGMA: Summarization/transformers/examples/seq2seq/finetune_trainer.py", line 351, in main
    resources_per_trial = {'gpu': 1})
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 1077, in hyperparameter_search
    best_run = run_hp_search(self, n_trials, direction, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/integrations.py", line 252, in run_hp_search_ray
    analysis = ray.tune.run(_objective, config=trainer.hp_space(None), num_samples=n_trials, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/tune.py", line 325, in run
    restore=restore)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/experiment.py", line 149, in __init__
    self._run_identifier = Experiment.register_if_needed(run)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/experiment.py", line 287, in register_if_needed
    register_trainable(name, run_object)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/registry.py", line 71, in register_trainable
    _global_registry.register(TRAINABLE_CLASS, name, trainable)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/registry.py", line 124, in register
    self.flush_values()
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/registry.py", line 146, in flush_values
    _internal_kv_put(_make_key(category, key), value, overwrite=True)
  File "/usr/local/lib/python3.6/dist-packages/ray/experimental/internal_kv.py", line 27, in _internal_kv_put
    updated = worker.redis_client.hset(key, "value", value)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 3050, in hset
    return self.execute_command('HSET', name, *items)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 900, in execute_command
    conn.send_command(*args)
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 726, in send_command
    check_health=kwargs.get('check_health', True))
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 718, in send_packed_command
    (errno, errmsg))
redis.exceptions.ConnectionError: Error 32 while writing to socket. Broken pipe.
02/01/2021 16:26:48 - INFO - wandb.sdk.internal.internal -   Internal process exited
```

## Expected behavior

The Trainer should run a hyperparameter search with the 8 different combinations of `learning_rate` and `gradient_accumulation_steps`.

Hey @marcoabrate it looks like you're seeing the same error as here https://github.com/huggingface/transformers/issues/9146. This should be fixed on transformers master and the latest Ray nightly wheels. Can you try with those and see if that fixes this? You can install the latest Ray nightly wheels by following the instructions here: https://docs.ray.io/en/master/installation.html#daily-releases-nightlies.<|||||>Hi @amogkam, thank you for your quick reply.
I am now on HF transformers master and I am installing raytune for Python 3.6 with

`pip install -U "https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-2.0.0.dev0-cp36-cp36m-manylinux2014_x86_64.whl"`

However, on Google Colab I still get the same error:

```
[INFO|trainer.py:358] 2021-02-02 11:04:25,269 >> Using amp fp16 backend
02/02/2021 11:04:25 - INFO - __main__ -   *** Hyperparameters Search ***
02/02/2021 11:04:25 - INFO - ray.tune.ray_trial_executor -   Initializing Ray automatically.For cluster usage or custom Ray initialization, call `ray.init(...)` before `tune.run`.
2021-02-02 11:04:26,687	INFO services.py:1182 -- View the Ray dashboard at http://127.0.0.1:8265
tcmalloc: large alloc 1236656128 bytes == 0x7fc271b46000 @  0x7fc54e359615 0x591e47 0x4cc179 0x4cc2db 0x566a71 0x5a4cd1 0x5a4fb8 0x7fc4a06ddc7c 0x7fc4a06e4bfa 0x7fc4a06e4fe2 0x7fc4a06e634f 0x7fc4a06e3a39 0x7fc4a06e4afc 0x7fc4a06e634f 0x7fc4a06e3a39 0x7fc4a06e4fe2 0x7fc4a06e634f 0x7fc4a06e3a39 0x7fc4a06e5d13 0x7fc4a06e643c 0x7fc4a06e3a39 0x7fc4a06e4f45 0x7fc4a06e67fa 0x7fc4a06e3a39 0x7fc4a06e5d5e 0x7fc4a06e643c 0x7fc4a06e3a39 0x7fc4a06e4f45 0x7fc4a06e67fa 0x7fc4a06e3a39 0x7fc4a06e5d5e
tcmalloc: large alloc 1545822208 bytes == 0x7fc215910000 @  0x7fc54e359615 0x591e47 0x4cc179 0x4cc2db 0x566a71 0x5a4cd1 0x5a4fb8 0x7fc4a06ddc7c 0x7fc4a06e4bfa 0x7fc4a06e4fe2 0x7fc4a06e634f 0x7fc4a06e3a39 0x7fc4a06e4afc 0x7fc4a06e634f 0x7fc4a06e3a39 0x7fc4a06e4fe2 0x7fc4a06e634f 0x7fc4a06e3a39 0x7fc4a06e5d13 0x7fc4a06e643c 0x7fc4a06e3a39 0x7fc4a06e4f45 0x7fc4a06e67fa 0x7fc4a06e3a39 0x7fc4a06e5d5e 0x7fc4a06e643c 0x7fc4a06e3a39 0x7fc4a06e4f45 0x7fc4a06e67fa 0x7fc4a06e3a39 0x7fc4a06e5d13
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 706, in send_packed_command
    sendall(self._sock, item)
  File "/usr/local/lib/python3.6/dist-packages/redis/_compat.py", line 9, in sendall
    return sock.sendall(*args, **kwargs)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/content/drive/My Drive/MAGMA: Summarization/transformers_last/transformers/examples/seq2seq/finetune_trainer.py", line 432, in <module>
    main()
  File "/content/drive/My Drive/MAGMA: Summarization/transformers_last/transformers/examples/seq2seq/finetune_trainer.py", line 346, in main
    resources_per_trial = {'gpu': 1})
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 1188, in hyperparameter_search
    best_run = run_hp_search(self, n_trials, direction, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/integrations.py", line 220, in run_hp_search_ray
    analysis = ray.tune.run(_objective, config=trainer.hp_space(None), num_samples=n_trials, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/tune.py", line 338, in run
    restore=restore)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/experiment.py", line 149, in __init__
    self._run_identifier = Experiment.register_if_needed(run)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/experiment.py", line 294, in register_if_needed
    register_trainable(name, run_object)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/registry.py", line 71, in register_trainable
    _global_registry.register(TRAINABLE_CLASS, name, trainable)
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/registry.py", line 124, in register
    self.flush_values()
  File "/usr/local/lib/python3.6/dist-packages/ray/tune/registry.py", line 146, in flush_values
    _internal_kv_put(_make_key(category, key), value, overwrite=True)
  File "/usr/local/lib/python3.6/dist-packages/ray/_private/client_mode_hook.py", line 47, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ray/experimental/internal_kv.py", line 35, in _internal_kv_put
    updated = worker.redis_client.hset(key, "value", value)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 3050, in hset
    return self.execute_command('HSET', name, *items)
  File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 900, in execute_command
    conn.send_command(*args)
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 726, in send_command
    check_health=kwargs.get('check_health', True))
  File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 718, in send_packed_command
    (errno, errmsg))
redis.exceptions.ConnectionError: Error 104 while writing to socket. Connection reset by peer.
02/02/2021 11:04:38 - INFO - wandb.sdk.internal.internal -   Internal process exited
```<|||||>I confirm with HF Transformers master and the latest ray[tune] version available using pip, the Trainer function works as expected.

Thank you for your help.