Runtime for BERT and Roberta
I'd like to train a BERT model from scratch. Approximately, how long should it take to train 800k sentences (batch size of say 32) on a 10GB GeForce RTX 2080 GPU? 

If I just fine-tune BERT for 800k sentences for 4 epochs, how long should that take?

Are there any benchmarks available except [exacct](https://blog.exxactcorp.com/nvidia-quadro-rtx-6000-bert-large-fine-tune-benchmarks-with-squad-dataset/)? 

How much faster is RoBerta?
Which version of bert or Roberta you want to use base, large ? It also depends on maximum sequence length<|||||>I'd like to use the base version with the maximum sequence length of 128.<|||||>Hi @AkshitaJha , you can run training for few steps 1 or 2, that should give you rough idea of how much time it'll take to finish one epoch<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
