All about the training speed in classification job 
I run the bert-base-uncased model with task 'mrpc' in ubuntu,nvidia p4000 8G. 
It's a classification problem, and I use the default demo data.
But the training speed is about 2 batch every second. Any problem?
I think it maybe too slow, but can not find why. I have another task with 1300000 data costs 6 hours per epoch.  
Maybe try to use a bigger batch size or try fp16 training?
Please refer to the [detailed instructions in the readme](https://github.com/huggingface/pytorch-pretrained-BERT#examples).