Fine-tuning (BERT & RoBERTa) base outperforms large
## ❓ Questions & Help

On all datasets I have used so far, base always outperforms large after the fine-tune. This is true for both BERT and RoBERTa. 

Why is that? Am I doing something wrong? Does large require far more epochs to train or a different learning rate?
Maybe you need more GPUs and a bigger batch size<|||||>I have 8 GPUs 2080 rtx, each with 10gb of data. But yeah I use a batch size of 4. However, I need the sequence length of 512 so it is impossible to increase the batch size.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
