XLM-R has extremely low accuracy after fine-tuning on MNLI
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Linux-4.14.138+-x86_64-with-debian-buster-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0a0+3bbb36e (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

### Who can help
 albert, bert, GPT2, XLM: @LysandreJik 


## Information

Model I am using (Bert, XLNet ...): XLM-R

The problem arises when using:
* [x] the official example scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)

## To reproduce

Steps to reproduce the behavior:

1. Run below command

```
python -m torch.distributed.launch \
    --nproc_per_node 8 transformers/examples/text-classification/run_glue.py \
    --model_name_or_path xlm-roberta-base \
    --task_name mnli \
    --do_train \
    --do_eval \
    --data_dir ../data/MNLI \
    --max_seq_length 128 \
    --per_device_train_batch_size 8 \
    --learning_rate 5e-5 \
    --num_train_epochs 2 \
    --save_steps 2000 \
    --output_dir xlmr_base_mnli \
    --overwrite_cache \
    --overwrite_output_dir
```
```
eval_mnli/acc = 0.31818644931227713
eval_mnli-mm/acc = 0.318246541903987
```

## Expected behavior
Higher performance on MNLI. Running script with `bert-base-multilingual-cased` instead yields around 80+ points, which was closer to the monolingual models' results.
~Someone mentioned that it could be due to the label indices being swapped for Roberta, but it seems there is already some  [code](https://github.com/huggingface/transformers/blob/57eb1cb68d1c567b25ac256444e5c1a77b8817a7/src/transformers/data/datasets/glue.py#L100) to deal with that.~ Tested Roberta and it works fine.<|||||>Tried running `run_xnli.py` with `xlm-roberta-base`  and got this error (seems like it could be related?):
```
08/06/2020 06:12:56 - INFO - transformers.data.processors.glue -   *** Example ***
08/06/2020 06:12:56 - INFO - transformers.data.processors.glue -   guid: train-5
08/06/2020 06:12:56 - INFO - transformers.data.processors.glue -   features: InputFeatures(input_ids=[0, 152725, 17, 14192, 398, 2367, 21208, 2174, 398, 738, 27167, 3060, 111, 8382, 69686, 148100, 17, 831, 1957, 15400, 5036, 398, 3714, 1836, 242, 107, 20949, 1257, 23, 70, 75281, 15437, 37457, 2, 2, 581, 69686, 148100, 765, 10, 37457, 111, 112034, 6, 5, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], attention_mask=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], token_type_ids=None, label=2)
08/06/2020 06:12:56 - INFO - __main__ -   Saving features into cached file /export/data/cached_train_xlm-roberta-base_128_xnli_en
Traceback (most recent call last):
  File "transformers/examples/text-classification/run_xnli.py", line 617, in <module>
    main()
  File "transformers/examples/text-classification/run_xnli.py", line 570, in main
    train_dataset = load_and_cache_examples(args, args.task_name, tokenizer, evaluate=False)
  File "transformers/examples/text-classification/run_xnli.py", line 343, in load_and_cache_examples
    all_token_type_ids = torch.tensor([f.token_type_ids for f in features], dtype=torch.long)
TypeError: an integer is required (got type NoneType)
```

@LysandreJik any idea what could be going wrong with XLM-R?<|||||>I don't think XLM-R is handled by the XLNI script. We should update it to the new trainer API and make these few lines model-agnostic.

https://github.com/huggingface/transformers/blob/45e26125de1b9fbae46837856b1f518a4b56eb65/examples/text-classification/run_xnli.py#L269-L273<|||||>I see, any idea what's going on with the GLUE/MNLI script? It seems to be using the new trainer API.<|||||>`run_pl_glue.py` has a similar issue in the dataloader.
```
  File "/export/multilingual-morpheus-analysis/transformers/examples/text-classification/lightning_base.py", line 155, in setup
    dataloader = self.get_dataloader("train", train_batch_size)
  File "text-classification/run_pl_glue.py", line 89, in get_dataloader
    all_token_type_ids = torch.tensor([f.token_type_ids for f in features], dtype=torch.long)
TypeError: an integer is required (got type NoneType)
```<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@samsontmr
Hi, i have the same problem with `xlm-roberta-large` on MNLI using the official script. `xlm-roberta-large` has extremely low accuracy. any suggestion?