Benchmark not replicable
## ❓ Questions & Help
Hello. I wanted to test if everything is allright with my downloads and so I ran the code snippet you provided in the section **Fine-tuning Bert model on the MRPC classification task** in the main README file (the only difference being the number of gpus-I use 4). However, my evaluation results are well below the ones you mention. I get 
```
acc = 0.8725490196078431
acc_and_f1 = 0.888829254329469
f1 = 0.9051094890510949
```
in my terminal.

There is also no output file in the specified folder.

Do you know what could cause this?

Thanks for answer
MS
Hello @Pointy-Hat 

Please tell us more, as explained [here](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md).<|||||>Technical data of the cluster used for computation:
System: CentOS Linux release 7.5.1804 (Core)
Python: 3.7.4
Pytorch: 1.3.1

Code run:
```
python3 -m torch.distributed.launch --nproc_per_node 4 ./examples/run_glue.py   \
    --model_type bert \
    --model_name_or_path bert-large-uncased-whole-word-masking \
    --task_name MRPC \
    --do_train   \
    --do_eval   \
    --do_lower_case   \
    --data_dir $GLUE_DIR/MRPC/   \
    --max_seq_length 128   \
    --per_gpu_eval_batch_size=8   \
    --per_gpu_train_batch_size=8   \
    --learning_rate 2e-5   \
    --num_train_epochs 3.0  \
    --output_dir /tmp/mrpc_single/ \
    --overwrite_output_dir   \
    --overwrite_cache 

```

Expected results (as per README.txt):

```
acc = 0.8823529411764706
acc_and_f1 = 0.901702786377709
f1 = 0.9210526315789473
```

Obtained results:
```
acc = 0.8725490196078431
acc_and_f1 = 0.888829254329469
f1 = 0.9051094890510949
```

GLUE data obtained via their `download_glue_data.py` script, as recommended in README.<|||||>did you try multiple random seeds?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
