question about tokenizer
hi, is there any difference between initializing BertTokenizer and load it using from_pretrained, 
such as:
1、tokenizer = BertTokenizer(bert-base-uncased-vocab)
2、tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

i want to pretrain for some new language by myself, so i need to create the vocabulary, and how could i integrate it to the transformers's tokenizer
thanks a lot!
This notebook might help you, it shows how you can train a LM from scratch on a new language.
https://github.com/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb<|||||>@patil-suraj thanks for your reply. i have read this notebook. but i want to re-use the functions in the transfomers such as functions like encode_plus, batch_encode_plus
so my question is :
after train a tokenizer by myself, how could i integrate it to the transformers's tokenizer to use functions in it?
thanks again<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
