Ok, so one would have to go to the source code and find the normalizers used and then replicate those? I guess this is a good starting point:
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased', use_fast=True)

tokenizer._tokenizer.normalizer

Output: <tokenizers.normalizers.BertNormalizer at 0x7fc7e939b2f0>

Would it be worth exposing more information about the normalizer used with the pretrained tokenizers (e.g. with a .normalizer or .config attribute)?
tokenizer.add_normalizers would be ideal, but I guess its not going to be built for an audience of one 
Just trying to think how to leverage the parallization power of the tokenizer with easy, off-the-shelf processing functions, without folks having to go digging into the source code
(cc @anthony)