Pretrain BERT with user defined vocabulary
# 🚀 Feature request

I'm wondering if there is a way to pretrain BERT with user-defined vocabulary, number of layers/heads, and use a WordLevel Tokenizer instead of a WordPiece Tokenizer current BERT uses. Thank you!

## Motivation

This could help the BERT mode to adapt to different tasks.


Hi! Yes, you can definitely do so (not sure about the word level). 

You can define a custom model configuration:
```py
from transformers import BertConfig

config = BertConfig(vocab_size=xxx, num_attention_heads=xxx, ...)
```

You can then use that configuration with the `run_mlm.py` script to train your model.

Regarding the tokenizer, you could use a word-level, but you would need to manually modify the files for that. There exists a word-level tokenizer in the `tokenization_bert.py` file that you could leverage for this.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> tokenization_bert.py

excuse me does this class have the functions that help in word-level, not sub-word level , please ? 