Have you seen this tutorial https://huggingface.co/transformers/custom_datasets.html#token-classification-with-w-nut-emerging-entities 54
For more about the tokenizer, see https://mccormickml.com/2019/07/22/BERT-fine-tuning/ 16
Chris McCormick also has a tutorial specifically about NER, but you’d have to pay for that one and I haven’t seen it.