run_glue.py with XLNet model on CoLA dataset reaches 0 accuracy
## Environment info
- `transformers` version: 4.1.1
- Platform: Linux
- Python version: 3.6.10
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
     
### Who can help
@TevenLeScao 

## Information
Model I am using: XLNet

The problem arises when using:
*  The official example scripts of `run_glue.py`

The tasks I am working on is:
* an official GLUE: CoLA

## To reproduce

Steps to reproduce the behavior:
I am using the "run_glue" cmd as described here: https://github.com/huggingface/transformers/tree/master/examples/text-classification
`python run_glue.py --task_name cola  --model_name_or_path xlnet-base-cased --do_train --do_eval --max_seq_length 128 --per_device_train_batch_size 32 --learning_rate 2e-5 --num_train_epochs 3.0 --do_predict --overwrite_output_dir`

That's the results I get: 
```
[p]$ cat res_cola_xlnet.txt 
eval_loss = 0.612945020198822
eval_matthews_correlation = 0.0
epoch = 3.0
```

## Expected behavior

Results > 0
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.<|||||>Hi, did you fix this issue? I also got this problem when using BART-large.