BERT model gets fairly random results
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.0
- Platform: linux
- Python version: 3.8
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): - 
- Using GPU in script?: - 
- Using distributed or parallel set-up in script?: - 

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@LysandreJik, @sgugger

## Information

Model I am using BERT-base-uncased model with run_glue.py on MRPC dataset, there are substantial differences in the results each time I run the codes, sometimes it reaches 5% percent. Too much variation makes the results not reliable, and this is quite a big issue. Thanks for your help on this. This might be a bug in the trainer/or the model itself.

first run:
```
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   epoch                        =                3.0
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_average_metrics         = 0.8300182784978398
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_cpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_cpu_peaked_delta    =                2MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_gpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_gpu_peaked_delta    =              264MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_accuracy           =              0.799
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_combined_score     =               0.83
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_f1                 =              0.861
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_loss               =             0.4643
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_runtime            =         0:00:00.38
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_samples_per_second =            529.617
```
second run:
```
[INFO|trainer_pt_utils.py:722] 2021-04-26 10:02:59,294 >> ***** test metrics *****
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   epoch                        =                3.0
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_average_metrics         = 0.8090236094437775
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_cpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_cpu_peaked_delta    =                2MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_gpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_gpu_peaked_delta    =              264MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_accuracy           =             0.7745
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_combined_score     =              0.809
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_f1                 =             0.8435
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_loss               =             0.4631
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,295 >>   mrpc_eval_runtime            =         0:00:00.35
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,295 >>   mrpc_eval_samples_per_second =            567.515
```

## To reproduce

Steps to reproduce the behavior:

Please run run_glue.py default script on MRPC.


## Expected behavior

The model needs to reproduce the same results each time it runs. 
Please give us the whole command you are running as we can't reproduce without it.
Are you properly setting the seed? Depending on the seed used, the results differ a lot on MRPC, since it's a tiny dataset. This is known and there have been [published papers](https://arxiv.org/pdf/2002.06305.pdf) on this.<|||||>Hi,
thank you the issue resolved with moving the codes to version 4.6.dev
thanks 