might even have to do changes to your clone of the original repository to only output the input_ids. Having written
a functional tokenization script that uses the original repository, an analogous script for 🤗 Transformers should be
created. It should look similar to this:
```python
from transformers import BrandNewBertTokenizer
input_str = "This is a long example input string containing special characters .$?-, numbers 2872 234 12 and words."
tokenizer = BrandNewBertTokenizer.from_pretrained("/path/to/tokenizer/folder/")
input_ids = tokenizer(input_str).input_ids
```
When both input_ids yield the same values, as a final step a tokenizer test file should also be added.
Analogous to the modeling test files of brand_new_bert, the tokenization test files of brand_new_bert should
contain a couple of hard-coded integration tests.
