My modified `run_glue.py` works well with v4.1.1 but not good with v4.6.0
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

`The environment in which the script doesn't work well`

- `transformers` version: 4.6.0
- Platform: Linux-5.4.0-72-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

`The environment in which the script works well`

- `transformers` version: 4.1.1
- Platform: Linux-5.4.0-72-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

Models:

- bert: @LysandreJik

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): bert-base-cased

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Here are some modified parts from the official `run_glue.py` example.
The version of the `run_glue.py` was for v4.1.1.
https://github.com/huggingface/transformers/blob/v4.1.1/examples/text-classification/run_glue.py

``` python

    # Preprocessing the datasets
    if data_args.task_name is not None:
        sentence1_key, sentence2_key = task_to_keys[data_args.task_name]
    else:
        # Again, we try to have some nice defaults but don't hesitate to tweak to your use case.
        non_label_column_names = [name for name in datasets["train"].column_names if name != "label"]
        if "sentence1" in non_label_column_names and "sentence2" in non_label_column_names:
            sentence1_key, sentence2_key = "sentence1", "sentence2"
        else:
            if len(non_label_column_names) >= 2:
                sentence1_key, sentence2_key = non_label_column_names[:2]
                if sentence2_key == "id" or sentence2_key == "idx":
                    sentence2_key = None
            else:
                sentence1_key, sentence2_key = non_label_column_names[0], None

    print(f"sentence1_key {sentence1_key}")
    print(f"sentence2_key {sentence2_key}")
```

``` python
    train_dataset = datasets["train"]
    eval_dataset = datasets["validation_matched" if data_args.task_name == "mnli" else "validation"]
    # if data_args.task_name is not None:
    #     test_dataset = datasets["test_matched" if data_args.task_name == "mnli" else "test"]
    test_dataset = datasets["test_matched" if data_args.task_name == "mnli" else "test"]

    # Log a few random samples from the training set:
    for index in random.sample(range(len(train_dataset)), 3):
        logger.info(f"Sample {index} of the training set: {train_dataset[index]}.")

    # Get the metric function
    if data_args.task_name is not None:
        metric = load_metric("glue", data_args.task_name)
    # TODO: When datasets metrics include regular accuracy, make an else here and remove special branch from
    # compute_metrics
```

``` python
    def compute_metrics(p: EvalPrediction):
        preds = p.predictions[0] if isinstance(p.predictions, tuple) else p.predictions
        preds = np.squeeze(preds) if is_regression else np.argmax(preds, axis=1)
        if data_args.task_name is not None:
            result = metric.compute(predictions=preds, references=p.label_ids)
            if len(result) > 1:
                result["combined_score"] = np.mean(list(result.values())).item()
            return result
        elif is_regression:
            # return {"mse": ((preds - p.label_ids) ** 2).mean().item()}

            # use the same metric as stsb (for pearsonr, spearmanr)
            metric = load_metric("glue", "stsb")
            result = metric.compute(predictions=preds, references=p.label_ids)
            return result            
        else:
            return {"accuracy": (preds == p.label_ids).astype(np.float32).mean().item()}
```

I tried to update my modified script referring to the latest version of `run_glue.py,` but it didn't solve the problem.

Steps to reproduce the behavior:

``` sh
(transformers4.1.1) $ CUDA_VISIBLE_DEVICES=0 python run_emobank_4.1.1.py \
   --model_name_or_path bert-base-cased \
   --train_file /path/to/train.csv \
   --validation_file /path/to/validation.csv \
   --test_file /path/to/test.csv \
   --do_train \
   --do_eval \
   --do_predict \
   --max_seq_length 64 \
   --per_device_train_batch_size 32 \
   --per_device_eval_batch_size 32 \
   --learning_rate 2e-5 \
   --num_train_epochs 10.0 \
   --load_best_model_at_end \
   --evaluation_strategy epoch \
   --metric_for_best_model eval_pearson \
   --output_dir /path/to/result/v4.1.1 \
   --overwrite_output_dir
```

If I run the script with transformers 4.1.1, it works well.
If 4.6.0, it runs without any error, but the result is not good.

`4.1.1 validation result`

```
eval_loss = 0.10065479576587677
eval_pearson = 0.6559863196369287
eval_spearmanr = 0.6244913632922552
epoch = 10.0
```

`4.6.0 validation result`

```
eval_loss = 0.16216666996479034
eval_pearson = 0.1785468733027603
eval_spearmanr = 0.18945952641568345
eval_runtime = 3.1986
eval_samples_per_second = 120.992
epoch = 10.0
```

## Expected behavior

Are there any tips to update my own script, which is written for v4.1.1, to be applied to v4.6.0+?

You should upgrade to 4.6.1, I think this is related to a bug fixed by #11785. Let us know if this doesn't solve your problem!<|||||>@sgugger 
Thank you for telling me the information!
I upgraded to 4.6.1 and tried running the script again, and now got the expected (or even better) result! 
(I expected to reproduce the result of 4.1.1 because I used the same hyperparameters, but the result was better than that.)

Thank you again!
