Exception during hyperparameter search with Ray and transformers library starting from version 4.5.0
I currently face the problem that with recent versions of the transformers library (issue starting at version 4.5.0)
the hyperparameter search with ray tune runs into a serialization issue described below.


## Environment info
- `transformers` version: 4.5.0
- Platform: Linux-4.19.0-16-amd64-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no
- Ray version: 1.4.0 

### Who can help
Maybe it is interesting to @richardliaw and @amogkam because they were mentioned as responsible for ray/raytune.

## Information

Model I am using (Bert, XLNet ...): distilbert-base-uncased ( model doesn't matter)

The problem arises when using:
* [ x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x ] an official GLUE/SQUaD task: (give the name): GLUE mrpc

## To reproduce

I have created a small working example which shows the error which (at least) I get:.
The code is mainly based on the [blog entry covering ray tune](https://huggingface.co/blog/ray-tune)

```python
import os
os.environ['TOKENIZERS_PARALLELISM'] = 'false'

from datasets import load_dataset, load_metric
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments
from ray import tune
from ray.util import inspect_serializability

model_name = 'distilbert-base-uncased'

tokenizer = AutoTokenizer.from_pretrained(model_name)
dataset = load_dataset('glue', 'mrpc')


def encode(examples):
    outputs = tokenizer(examples['sentence1'], examples['sentence2'], truncation=True)
    return outputs

encoded_dataset = dataset.map(encode, batched=True)

def model_init():
    return AutoModelForSequenceClassification.from_pretrained(model_name, return_dict=True)

def compute_metrics(eval_pred):
    metric = load_metric('glue', 'mrpc')
    predictions, labels = eval_pred
    predictions = predictions.argmax(axis=-1)
    return metric.compute(predictions=predictions, references=labels)

training_args = TrainingArguments("test")

trainer = Trainer(
    args=training_args,
    tokenizer=tokenizer,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    model_init=model_init,
    compute_metrics=compute_metrics,
)

def search_params(trial):
    return {
        #toy example
        "learning_rate": tune.grid_search([0.000001, 0.00001, 0.0001, 0.001]),
    }
    
trainer.hyperparameter_search(
    direction="maximize", 
    backend="ray", 
    hp_space = search_params,
    n_trials=1,
)
```

This code snippet works with transformers version 4.4.2 and ealier but not on versions 4.5.0 and later.

The error which appeared is
```python
Traceback (most recent call last):
  File "working_example.py", line 48, in <module>
    trainer.hyperparameter_search(
  File "/site-packages/transformers/trainer.py", line 1459, in hyperparameter_search
    best_run = run_hp_search(self, n_trials, direction, **kwargs)
  File "/site-packages/transformers/integrations.py", line 231, in run_hp_search_ray
    analysis = ray.tune.run(
  File "/site-packages/ray/tune/tune.py", line 297, in run
    _ray_auto_init()
  File "/site-packages/ray/tune/tune.py", line 664, in _ray_auto_init
    ray.init()
  File "/site-packages/ray/_private/client_mode_hook.py", line 62, in wrapper
    return func(*args, **kwargs)
  File "/site-packages/ray/worker.py", line 866, in init
    hook()
  File "/site-packages/ray/tune/registry.py", line 171, in flush
    self.references[k] = ray.put(v)
  File "/site-packages/ray/_private/client_mode_hook.py", line 62, in wrapper
    return func(*args, **kwargs)
  File "/site-packages/ray/worker.py", line 1527, in put
    object_ref = worker.put_object(value)
  File "/site-packages/ray/worker.py", line 280, in put_object
    serialized_value = self.get_serialization_context().serialize(value)
  File "/site-packages/ray/serialization.py", line 326, in serialize
    return self._serialize_to_msgpack(value)
  File "/site-packages/ray/serialization.py", line 306, in _serialize_to_msgpack
    self._serialize_to_pickle5(metadata, python_objects)
  File "/site-packages/ray/serialization.py", line 266, in _serialize_to_pickle5
    raise e
  File "/site-packages/ray/serialization.py", line 262, in _serialize_to_pickle5
    inband = pickle.dumps(
  File "/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
    return Pickler.dump(self, obj)
TypeError: cannot pickle '_thread.RLock' object
```

Based on this error, I searched for code to check which part is not serializable (because the whole trainer is transferred to each ray trial). I found the [ray serialization page](https://docs.ray.io/en/master/serialization.html#troubleshooting) and executed
```python
inspect_serializability(trainer, name="test")
```
The output was:

```
================================================================================
Checking Serializability of <transformers.trainer.Trainer object at 0x7fce1cbbeee0>
================================================================================
!!! FAIL serialization: cannot pickle '_thread.RLock' object
    Serializing 'compute_metrics' <function compute_metrics at 0x7fce1cb5b5e0>...
    Serializing 'model_init' <function model_init at 0x7fce1cb5b550>...
    Serializing '_gather_and_numpify' <bound method Trainer._gather_and_numpify of <transformers.trainer.Trainer object at 0x7fce1cbbeee0>>...
    !!! FAIL serialization: cannot pickle '_thread.RLock' object
        Serializing '__func__' <function Trainer._gather_and_numpify at 0x7fce1f739940>...
    WARNING: Did not find non-serializable object in <bound method Trainer._gather_and_numpify of <transformers.trainer.Trainer object at 0x7fce1cbbeee0>>. This may be an oversight.
================================================================================
Variable:

        FailTuple(_gather_and_numpify [obj=<bound method Trainer._gather_and_numpify of <transformers.trainer.Trainer object at 0x7fce1cbbeee0>>, parent=<transformers.trainer.Trainer object at 0x7fce1cbbeee0>])

was found to be non-serializable. There may be multiple other undetected variables that were non-serializable.
Consider either removing the instantiation/imports of these variables or moving the instantiation into the scope of the function/class.
If you have any suggestions on how to improve this error message, please reach out to the Ray developers on github.com/ray-project/ray/issues/
================================================================================
```

I did not find any major changes between version 4.4.2 and 4.5.0 with regards to integrations.py and trainer.py.

I think the first step would be, that someone else reproduce the behaviour if possible (maybe something is also wrong on my side/setup).

Hey @sven-h yes this is a known issue, same as https://github.com/huggingface/transformers/issues/11249.

From this thread:

> If you disable the memory tracker (pass in skip_memory_metrics=True into your TrainingArguments) then you will no longer get the pickling error. In the next transformers release, the Ray Tune integration will automatically disable memory tracking if it's currently being enabled.
<|||||>Hi @amogkam
thanks for the fast reply and the answer.