Not getting the same results with run_qa and run_qa_no_trainer scripts
## Environment info
Just followed the default setup instructions in a new conda environment:
```shell
git clone https://github.com/huggingface/transformers
cd transformers
pip install .
pip install -r examples/pytorch/question_answering/requirements.txt
```

### Who can help
@sgugger @patil-suraj 

## Information
Model I am using: https://huggingface.co/prajjwal1/bert-tiny

My goal is to run the finetuning example on the bert-tiny model and Squad dataset, with and without the Trainer class, and to obtain the same results.

## The problem
Running with the Trainer class with:
```shell
CUDA_VISIBLE_DEVICES=0 python run_qa.py \
  --model_name_or_path prajjwal1/bert-tiny \
  --dataset_name squad \
  --do_train \
  --do_eval \
  --per_device_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /trainer_squad/ 
```
ends with: `eval_exact_match = 31.3245` and `eval_f1 = 43.3655`.

Then, running the same setup but without the Trainer with:
```shell
CUDA_VISIBLE_DEVICES=0 python run_qa_no_trainer.py \
  --model_name_or_path prajjwal1/bert-tiny \
  --dataset_name squad \
  --per_device_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /no_trainer_squad/
```
ends with: `Evaluation metrics: {'exact_match': 19.981078524124882, 'f1': 32.57782310536579}`.

It's interesting that I obtain the same results (`F1 = 49.73` and `EM = 48.6`) when I run with and without Trainer class, but with a different dataset: `--dataset_name squad_v2` and `--version_2_with_negative`.
The two scripts are different, they do not have the same defaults for the `seed` and even then, they do not randomize the data the same way. It's impossible for the two of them to give you the same results. <|||||>Do you have any suggestions on what to modify in the `run_qa_no_trainer.py` to get a bit better results (possibly closer to "the baseline" with `run_qa.py`)? 
Based on the same results I got with the `squad_v2`, I assumed that both scripts are doing the same thing, and that the one without the Trainer just provides more flexibility to modify stuff in the train/eval loops. <|||||>On one GPU, you could maybe achieve the same results by passing the same seeds, making sure it's set at the right place, but that's a big maybe. Reproducibility is a hard enough problem with one script, so with two scripts that start with different assumptions and use different APIs, it's next to impossible.<|||||>Okay, got it! I then just misunderstood their roles in the examples project. Thanks for a quick reply.