Hey, I am trying to add subword tokens to bert base uncased as follows:

num = tokenizer_bert.add_tokens(’##committed’)


tokenizer_bert.tokenize(’##committed’)
[’##committed’]
tokenizer_bert.tokenize(‘hellocommitted’)
[‘hello’, ‘##com’, ‘##mit’, ‘##ted’]

It seems like the tokenizer is literally adding the hashtags, when I would want to create a new subword called ##commited. I am doing this to deal with hashtags & thinking of initializing those new subwords to their original words.
Any solutions / better ways to deal with hashtags, would be really appreciated!
Thanks –