How to pretrain my own data with this pytorch code?
I wonder how to pretrain with my own data.
Sir @Gpwner  i think you have to refer to do pretrain for google/bert repo( https://github.com/google-research/bert#pre-training-with-bert ) and then convert tensorflow model as pytorch.<|||||>A pre-training script is now included in `master` thanks to @tholor's PR #124 <|||||>> A pre-training script is now included in `master` thanks to @tholor's PR #124
Thanks ！！
Does it support Multiple GPU？Because the official script does not support Multiple GPU<|||||>It does (you can read more about it [here in the readme](https://github.com/huggingface/pytorch-pretrained-BERT#lm-fine-tuning))<|||||>> It does

great job，thanks ～<|||||>All thanks should go to @tholor :-)