OK, thanks for replying.
What happens if you do tokenizer.tokenize(‘hellocommitted’)
immediately after doing tokenizer.tokenize(‘helloxxxcommitted’)
What is the result of tokenizer.tokenize(‘uncommitted’)
Clearly, it has managed to create the ‘committed’ token, so it seems very odd that it wouldn’t use it except when it arrives with helloxxx.  Is that really happening, or is it just that your earlier attempt failed for some reason, ie when you did
tokenizer.add_tokens([   ‘committed’   ])
Maybe it doesn’t like the spaces, or maybe there was a system failure at the wrong moment.
I just noticed something else odd: your last example should have output ‘helloxxx’, ‘##committed’ (if the ‘committed’ token has been added at all).   Are you sure that’s what it said?
On a different tack, what do you mean by “deal with hashtags”  ?   Are you using twitter data?  What does your bert model currently do with # inputs?   Are you sure that that wouldn’t be fine?  Do you have enough data to retrain the model to deal with the new ‘committed’ token?  (see the answer to this post  https://stackoverflow.com/questions/60068129/transformers-pretrainedtokenizer-add-tokens-functionality 22 ).   Could you preprocess all your input data so that the input # characters become something else that doesn’t have a special meaning within transformers?