You can use this function (tf.strings.regex_replace) 1 if you want to manipulate texts using regular expressions in the TF graph.
SentencePiece and WordPiece tokenizers are useful for me. I recommend these tokenizers to you.

text.pad_model_inputs 1 is very useful to pad model inputs.
I don’t know what task you want to solve, so I’m just guessing the task. If you are solving a sequence tagging task, I think you can use tensorflow-text’s tokenizers and use tokenize_with_offsets to get offsets of each token. Then, you can use those offsets to map probabilities to texts. (For example, you can use tokenize_with_offsets in WordpieceTokeizer 1)