TFBertTokenizer - support for "never_split"
### Feature request

Often vocabularies contain special tokens that should not be split.

For example, in model "anferico/bert-for-patents", the vocabulary contains a token "[abstract]" (token_id is 5)
https://huggingface.co/anferico/bert-for-patents/raw/main/vocab.txt

The normal `BertTokenizer` supports a param "never_split" for this:

```python
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('anferico/bert-for-patents', never_split=['[abstract]'])
tokenizer.tokenize('[abstract] some text here')
# ['[abstract]', 'some', 'text', 'here']
```

So above, even though '[abstract]' has parens, it is not split.

But `TFBertTokenizer` does not have a "never_split" param, and so there is no way to prevent splits. For example:

```python
from transformers import TFBertTokenizer
tokenizer = TFBertTokenizer.from_pretrained('anferico/bert-for-patents')
tokenizer(tf.constant(['[abstract] some text here']))
# {'input_ids': <tf.Tensor: shape=(1, 8), dtype=int64, numpy=array([[   2, 1036, 9726, 1038, 1735, 3458, 1847,    3]])>, 'attention_mask': <tf.Tensor: shape=(1, 8), dtype=int64, numpy=array([[1, 1, 1, 1, 1, 1, 1, 1]])>, 'token_type_ids': <tf.Tensor: shape=(1, 8), dtype=int64, numpy=array([[0, 0, 0, 0, 0, 0, 0, 0]])>}
```

Above, notice that token_id 5 (['abstract']) is missing in the input_ids, and in fact '[abstract]' has been split into three separate tokens:
* '[' - 1036
* 'abstract' - 9726
* ']' - 1038


### Motivation

We would like to use an end-to-end model, on TensorFlow Serving, with in-graph tokenization.

But we need to be able to include special tokens in our input, such as '[abstract]', '[claims]' etc
https://huggingface.co/anferico/bert-for-patents/raw/main/vocab.txt

If TFBertTokenizer had a "never_split" param, this would be possible.

But currently it is not, so we need to do Tokenization outside the server.

cc @Rocketknight1 <|||||>Hi @benzitohhh , and sorry for the delay! This is an interesting and useful idea, but we're depending on the underlying Tensorflow Text layers, specifically [BertTokenizer](https://www.tensorflow.org/text/api_docs/python/text/BertTokenizer) in this case.

I don't think there is a 'never_split' option here, but we could use the `preserve_unused_token` argument. This would mean that tokens of the form `[unused0]`, `[unused1]`, etc. would never be split, so you could use those as a control token like `[abstract]`. Would this work for your use case? If it's useful to you it's probably useful to other people, and we can add it to the `TFBertTokenizer` layer in a PR.<|||||>hi @Rocketknight1 Thanks for the response, and sorry so slow getting back to you also!

Just to check I understand...

In our case, vocabulary (token_id to token mapping) looks as below, where 5-9 inclusive are "special" tokens:
https://huggingface.co/anferico/bert-for-patents/raw/main/vocab.txt

```
0: [PAD]
1: [UNK]
2: [CLS]
3: [SEP]
4: [MASK]
5: [abstract]
6: [claim]
7: [summary]
8: [invention]
9: [cpc]
10: [unused0]
11: [unused1]
12: [unused2]
13: [unused3]
14: [unused4]
15: [unused5]
etc...
```

So with the `preserve_unused_token` approach, I guess we'd need to do something like:

```python
input = '         [abstract]   some      text      here.       '
#out  = [2,       5,           1735,     3458,     1847,      3]  #### Expected tokenized ids

# 1. Replace each "special" token with a unique "unused" token
# So we need to map:
#    '[abstract]' -> '[unused0]'
#    '[claims]'   -> '[unused1]'
# etc..
# I guess we could use some regex for this.
input__unused = '[unused0] some text here'

# 2. Do the tokenization
bert_input__unused = tokenizer(tf.constant([input__unused]))
# { 'input_ids': ... numpy=array([[   2, 10, 1735, 3458, 1847,  3]])> etc... }
# i.e. the "10" above is the is '[unused0]' token

# 3. Replace "unused" token_ids with the correct special token_ids
# Not sure exactly how to do this with tensor operations, but I guess it's possible?
# So we need to map:
#    10 ('[unused0]') -> 5 ('[abstract]')
#    11 ('[unused1]') -> 6 ('[claims]')
# etc..
bert_input = ..
# { 'input_ids': ... numpy=array([[   2, 5, 1735, 3458, 1847,  3]])> etc... } 
```

Will the above work?

If so, that would be amazing, and totally solve our situation.

Obviously, being able to add a "never_split" param would be much nicer :)

Anyways let us know what is possible - thanks!<|||||>Hi @benzitohhh, yes, that's correct! I'd also need to file a PR to expose the option in our tokenizer, but if you're interested then I can do that soon.

For the issue of mapping the `unused` token ids to the correct special token IDs, I suggest using a block of unused token IDs in the same order as your special token IDs. Then all you would need to do is:

```python
# `True` for `unused` tokens, `False` otherwise
condition = (input_ids >= unused_start_idx) & (input_ids <= unused_end_idx)
# Subtract offset value from all unused tokens
input_ids = tf.where(condition, input_ids - offset, input_ids)
```
In the vocab list you linked above, an offset of `5` would map `[unused0]` -> `[abstract]` and so on.<|||||>For more complex replacements, you could also just reorder the `vocab_list` for the `TFBertTokenizer` so it generates the indices you want!<|||||>@Rocketknight1 Ok this would totally work for us, and would allow us to create an end-to-end model - yay!

If you could create a PR that would be super appreciated.

Thanks again for all your help here, and the super clear explanations. Have a good weekend meanwhile.

<|||||>Hi @benzitohhh, the PR is now open at #24324. You can try out the PR branch with the following command:
```
pip install git+https://github.com/huggingface/transformers.git@allow_tf_tokenizer_kwargs
```

When creating the `TFBertTokenizer`, add the arguments `use_fast_bert_tokenizer=False` and `preserve_unused_token=True`. Also, note that only the slower TF tokenizer layer supports the `preserve_unused_token` argument, but only the fast layer can be exported to TFLite. This means that this solution won't work for you if you want to export to TFLite! <|||||>@Rocketknight1 Ah amazing thanks! Will try this out first thing on Monday and let you know asap<|||||>@Rocketknight1 Ok just tested the PR, it works perfectly.

Thanks again for making this happen!<|||||>Cool! Hopefully we can merge the PR soon in that case, so you can stop installing from the branch.<|||||>@benzitohhh this has now been merged. You can now get it just by installing from `main` with
```
pip install git+https://github.com/huggingface/transformers.git
```
It will be included with the next release of transformers in a few weeks, at which point you can go back to the usual `pip install transformers`<|||||>@Rocketknight1 amazing - thanks again