morgan:

Or if I added normalizer to a pretrained tokenizer and then call the tokenizer with datasets, will that also carry out the normalization before doing the tokenization?


Yes that is exactly the way to go. To get maximum efficiency you want to avoid round trips between python and Rust as much as possible. We are in the process of getting a documentation out that will hopefully improve that.
Just a gist here, the de-facto method to create a new Tokenizer in 0.9.0 is:
from tokenizers import Tokenizer, models, normalizers, ....

tokenizer = Tokenizer(models.BPE())
tokenizer.normalizer = normalizers.NFKC()
tokenizer.pre_tokenizer = ... # some pre tokenizer
tokenizer.decoder = ...
tokenizer.processor = ...
etc...

# Optional training phase, not required 
# if you loaded from an existing model.

trainer = trainers.BpeTrainer(vocab_size=80000
tokenizer.train(trainer, ["myfile.txt"])

# Training

tokenizer.encode("my string").tokens


Then everything will be run within rust, and should get you the best speedup possible.
If you save your tokenizer with tokenizer.save('mytokenizer.json') it should save the whole tokenization pipeline. Its another way of checking the various options of existing tokenizers.