Exact and F1 score do not increase when fine-tunes XLM on the SQuAD dataset
## ❓ Questions & Help

I am trying to fine-tune XLM on the SQuAD dataset.
The command is as following:
[CUDA_VISIBLE_DEVICES=0 python run_squad.py --model_type xlm --model_name_or_path xlm-mlm-tlm-xnli15-1024 --do_train --do_eval --train_file $SQUAD_DIR/train-v1.1.json --predict_file $SQUAD_DIR/dev-v1.1.json --per_gpu_train_batch_size 12 --learning_rate 3e-5 --num_train_epochs 100 --max_seq_length 384 --doc_stride 128 --eval_all_checkpoints --save_steps 50 --evaluate_during_training --output_dir /home/weihua/Sqad/transformers/xlm_out/ ]
![WeChat Image_20191112113346](https://user-images.githubusercontent.com/43492059/68640277-2da52700-0542-11ea-981b-c748e918e1d8.png)

All the parameters I set are in accordance with the example in the pytorch-transformers document. But Exact and F1 scores have barely increased. And the loss of each epoch also drops very slowly. Each epoch drops by about 0.01.
![68477603-d6b10080-0268-11ea-947f-995b157da8d3](https://user-images.githubusercontent.com/43492059/68640296-3f86ca00-0542-11ea-93b3-db1bd64539b2.png)


Is there a problem with my parameter settings? Or do I need to adjust some parts of the model when I use XLM?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>duplicate #1799 