Pretrain From Scratch using Google TPU
@julien-c @patrickvonplaten 
I want to pretrain a model from scratch by utilizing Google Cloud TPU offered in kaggle. I can train the model without TPU but I want to train it on a TPU. Any help will be much appreciated.

Also what options do I have, if there is no  straight forward approach ?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
