Can't reproduce xlm-roberta-large finetuned result on XNLI
# ❓ Questions & Help

I'm trying to finetune `xlm-roberta-large` on MNLI English training data and make zero-shot classification on XNLI dataset.
However, I found that `xlm-roberta-large` is super sensitive to hyper parameters. The reported average accuracy is 80.9, while my model can only achieve 79.74, which is 1% less than the reported accuracy. 

I used Adam optimizer with 5e-6 learning rate and the batch size is 16.  Any one can suggest better hyperparameters to reproduce the XNLI result of `xlm-roberta-large`?

Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!