how can I pre-training my own model from the existed model or from scratch
## ❓ Questions & Help

I want to load the pre-training model like bert offered by google,and train language model on more corpus,how can I do it ?thanks
Hi, you can see how to use the library in the [documentation](https://huggingface.co/transformers/). You might be interested in the library philosophy and the way to load pre-trained models, which is [described here](https://huggingface.co/transformers/quickstart.html). You might also be interested in the [examples](https://huggingface.co/transformers/examples.html), which showcase [how to fine-tune a language model](https://huggingface.co/transformers/examples.html#language-model-fine-tuning).<|||||>@hischen did you find solution for pre-training BERT on your corpus?
@LysandreJik fine tuning is different from pre-training. I could not find documentation about pre-training the model on a corpus. Can you please help me with that.

Regards,
D. Ravi Theja.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
