Hello @nabito,
This tutorial might interest you: https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/tokenizer_training.ipynb 2
I didn’t train the tokenizer for BigBird while adding it to Transformers & rather used this vocabulary file directly: bigbird/bigbird/vocab at master · google-research/bigbird · GitHub
Possibly, the colab link (I shared above) will help you in training BigBirdTokenizerFast on your dataset.