Changes to S3 Roberta / RobertaForSequenceClassification
Hello,
I'm wondering if there have been any changes in either the pretrained Roberta model or the configuration of RobertaForSequenceClassification within the past month or so. I am initializing it as `RobertaForSequenceClassification.from_pretrained(...)` and running it as demonstrated in `run_glue.py`.

For a custom dataset, I am noticing that the fine-tune accuracy has decreased by several percentage points on newly trained models as opposed to ~ 1 month ago. This happens consistently (i.e I've tried retraining multiple times) and using the same hyperparams.

In addition, I've noticed that the new training times are cut by ~1/2, so the model seems to train faster, but is less performant.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
