May I get the details of Bert pre-train procedure?
## ❓ Questions & Help
I want to measure the time consume in BERT pre-train procedure. May I ask some questions here:
1. What is the pre-train data? Is that same in paper : BooksCorpus & English Wiki.
2. Is there some code for pre-train can I utilize?
3. May I know how much epochs should I train when pre-train 'base' version model or 'large' version model?
Hi, you should refer to the original Bert repository and papers for details on the pretraining: https://github.com/google-research/bert<|||||>Thank you! Get it.