Bad result in fine-tuning XLNet for SQuAD
Hello,

I'am fine-tuning XLNet for SQuAD V1.1 task, but I get the bad result. I got checkpoint of XLNet in [model hub](https://huggingface.co/xlnet-base-cased).

GPU: single GeForce RTX 3090 24G

running scripy:
`CUDA_VISIBLE_DEVICES=4 python ./examples/pytorch/question-answering/run_qa.py   --model_name_or_path ../pretrained_model/xlnet_base   --dataset_name squad   --do_train   --do_eval   --per_device_train_batch_size 12   --learning_rate 3e-5   --num_train_epochs 2   --max_seq_length 384   --doc_stride 128   --overwrite_output_dir   --output_dir ../squad/xlnet_base`

result:
`{
    "epoch": 2.0,
    "eval_samples": 10848,
    "exact_match": 12.639545884578997,
    "f1": 14.638577161480404,
    "train_runtime": 7179.3726,
    "train_samples": 88835,
    "train_samples_per_second": 2.062
}`

Hi @Timothy023 

Please use the [forum](https://discuss.huggingface.co/) to post such questions. Thanks!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.