Hello @nabito,
This issue is similar to Text Classification on GLUE on TPU using Jax/Flax : BigBird · Issue #12483 · huggingface/transformers · GitHub 1.
Tokenizer config needs to be updated from HuggingFace Hub (while Tokenizer code is absolutely fine) & I need @patrickvonplaten’s approval for updating that.
@patrickvonplaten, fix is quite simple & we just need to run this:
wget https://huggingface.co/google/bigbird-roberta-base/resolve/main/spiece.model

from transformers import BigBirdTokenizer
tokenizer = BigBirdTokenizer("spiece.model")
tokenizer.push_to_hub("google/bigbird-roberta-base")

@nabito, for now you can do this:
wget https://huggingface.co/google/bigbird-roberta-base/resolve/main/spiece.model

from transformers import BigBirdTokenizer
tokenizer = BigBirdTokenizer("spiece.model")

# similarly for fast tokenizer