hey @Joanna if i understand correctly, your question is fundamentally about the manner in which we tokenize raw text into tokens right? for example, BERT uses a WordPiece tokenizer that can be used for all downstream tasks, provided some alignment between the tokens and labels is given for tasks like NER.
there’s a nice walkthrough the various tokenization strategies that are employed in transformers here: Summary of the tokenizers — transformers 4.5.0.dev0 documentation 2