added tokens may split a normal token into halves
In the tokenizer base class, split_on_token() attempts to split input text by each of the added tokens. Because it uses text.split(tok), it may accidentally split a token at the middle. 

For example a new token "ht" is added to the vocabulary. Then "light" will be split into "lig" and "". But as "light" is in the original vocabulary, it should be left intact to be processed by self._tokenize(). 

Hence I'd suggest to replace it with re.split, which will split only at word boundaries ([0-9a-zA-Z_]). 

But in languages whose word boundaries are different from English, this behavior may be undesirable and the user can revert to the old text.split(). It is controlled by a newly added flag split_added_on_word_boundary. 

This looks great, thanks a lot @askerlee.

Can you add a docstring for the new argument?

Ideally also a test in [tokenization_tests_commons.py](https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/tests/tokenization_tests_commons.py)<|||||>Thanks @thomwolf . I updated my patch to make it neater. Will close this pull request and submit a new request soon.