OK, thanks Bram, I haven’t looked at sentencepiece tokenizing.
My wordpiece estimates were based on samples from my specific set of texts.
Katarina: this page of the huggingface documentation discusses the different kinds of tokenizers Summary of the tokenizers — transformers 4.3.0 documentation and this blog looks like it might be useful Tokenizers: How machines read