In your code snippet you are creating th character dictionary and mapping via the tokens input so an index will be assigned to each character by iterating it as seen here:
tokens = ['c', 'a', 'z', '0', 'b']

chars = tokens
int2char = dict(enumerate(chars))
char2int = {ch: ii for ii, ch in int2char.items()}
# > {'c': 0, 'a': 1, 'z': 2, '0': 3, 'b': 4}}