If anything I’d expect the ‘_’ token to appear before rather than after ‘[MASK]’. i.e
from transformers import AlbertTokenizer

tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')
inputs = tokenizer.encode("The capital of France is Paris.")
print(inputs)
# [2, 14, 1057, 16, 714, 25, 1162, 9, 3]
print(tokenizer.convert_ids_to_tokens(inputs))
# ['[CLS]', '▁the', '▁capital', '▁of', '▁france', '▁is', '▁paris', '.', '[SEP]']
print(tokenizer.decode(inputs))
# [CLS] the capital of france is paris.[SEP]

I suspect it’s something to do with the fact that [MASK] is a special, all other word tokens will start with _. Perhaps there’s a bug where ’ [MASK]’ maps to ‘[MASK]’, ‘_’ rather than ‘_’ ,’[MASK]’?
Or perhaps it’s just not supported (tokenising text which actually contains literal specials?)