Hi @nielsr  Thank you so much for the effort on the notebooks! I was following your tutorials. Just a follow-up question. In my customized dataset I also got many broken subwords as @AndreaSottana mentioned. I was just wondering, what if I design a new tokenizer that can handle my dataset to avoid the subwords problem caused by WordPiece, then retrain BERT model on my customized tokenizer and dataset. Will this kinda solve the problem for NER labeling?   The intuition is by doing this  I can reduce the number of subwords to make BERT learn my domain context better.  I am guessing there might be some issues with the WordPiece tokenizer (where the BERT were original trained) and my customized tokenizer working on my customized datasets. Any ideas or comments?