customize special tokens
# ❓ Questions & Help

## Details

I am trying to add more special tokens in order to train new language models without modifying the model architecture but simply modifying the input data. For example, adding more separators to involving more complex structure information. 

Is there a simple, clear, consistent way to add/customize special tokens. 
1. The function `add_special_tokens` can only add token string to token attribute specified in `SPECIAL_TOKENS_ATTRIBUTES`
2. I tried to extend this list and `add_special_tokens` works, but calling 
`tokenizer.get_special_tokens_mask` report that _XXX does not exist, where XXX is the new customized special token attribute

finally I found a solution: modifying `tokenizer._additional_special_tokens.extend` directly makes `tokenizer.get_special_tokens_mask` work. But I don't know how could this work and if there is any other side effect. 

Properties like `special_tokens_map` is also confusing, it seems like a @property implementation, which is not modifiable. And I don't know where this could be used to have what effect. 

I cost much time to read the source code but still lack a overview about the special token architecture in tokenizers. 
For special tokens that don't correspond to one of the attributes of tokenizers, you should pass them to `add_special_tokens` with the `additional_special_tokens` keyword arguments:
```
tokenizer.add_special_tokens(additional_special_tokens = ['TK1', 'TK2' ...])
```
You can then change `tokenizer.additional_special_tokens` directly if you need to add or remove some of those tokens.<|||||>@sgugger Thank you very much.