Can't Reproduce GLUE scores using official BERT implementation
### System Info

- `transformers` version: 4.32.0.dev0
- Platform: Linux-5.15.0-1036-aws-x86_64-with-glibc2.31
- Python version: 3.9.16
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

As stated at https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification, the scores don't match the official scores on paper and GLUE benchmark. My scores match the huggingface benchmark but is lower than the official implementation in some benchmarks like CoLA. How did this happen and how can we avoid it? Looking forward to your help!

<img width="1018" alt="image" src="https://github.com/huggingface/transformers/assets/60613238/5f368343-0619-4084-9039-39683f772d3f">


### Expected behavior

<img width="550" alt="image" src="https://github.com/huggingface/transformers/assets/60613238/b6cc374a-5e2d-409d-ac9d-3b73a7ced4fd">

The results you will get are very sensitive to the seed used.<|||||>Hi @sgugger , so it means that as long as I get the same results as on the huggingface README, my model is nearly identical to the paper, right?
Also a follow-up: in our paper we shall also include this score. Do you remember anyone ever reproduced the scores in the paper using huggingface?<|||||>I remember getting the same results on Cola or even better results on Cola with a different seed (but then some of the other results were different). I don't remember which seed however :sweat_smile: <|||||>Ugh that's embarassing 😅 these original authors are too good at these kind of tricks.
Anyway, thanks for the instant help! I'll get back to this thread if I get identical or even better results.
<|||||>@sgugger Sorry for putting this up after closing the issue. I'm writing to ask if you know anyone successfully reproduced pre-training the RoBERTa model from scratch and gained as good scores as the listed ones as shown in https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification. Specifically, I'm interested in how the script they use with [`run_mlm.py`](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_mlm.py). 
Looking forward to your reply!<|||||>`run_mlm` is just an example, it does not reproduce RoBERTa pretraining.