Words after tokenization replaced with #
Hello,

When training the bert-base-multilingual-cased model for Question and Answering, I see that the tokens look like this : 

```tokens: [CLS] what is the ins ##ured _ name ? [SEP] versi ##cherung ##ss ##che ##in erg ##o hau ##srat ##versi ##cherung hr - sv 927 ##26 ##49 ##2 ```

Any idea why words are getting replaced with #? 

Here is the command I am using : 

```python run_squad.py --bert_model bert-base-multilingual-cased --do_train --do_predict --train_file dataset_train.json --predict_file dataset_predict.json --train_batch_size 12 --learning_rate 3e-5 --num_train_epochs 2.0 --max_seq_length 400 --doc_stride 20 --output_dir output_dir```
Because it uses WordPiece tokenization, and will introduce the `#` token.
Check: https://github.com/google-research/bert#tokenization<|||||>@ymcui okay sweet, thank you. Will use the relevant one. <|||||>@ymcui How do I change this ? or is not possible to do so?<|||||>1. If you are training completely from scratch, then it will be possible to use your own tokenizer.
2. However, if you are fine-tuning on the existing pre-trained BERT models, I think it will not be possible to change the tokenizer, as the pre-trained BERT models are trained using WordPiece tokenizer. <|||||>@ymcui is right.

Since the purpose of the present repo is to supply pre-trained model basically you are stuck with WordPiece tokenization.

If you build a new model and train it from scratch, you can selected whatever tokenization you want :-)<|||||>@ymcui @thomwolf - Yes, that is quite a problem and thanks for getting back. Evaluating building something on our own now 🗡 