BertTokenizerFast does not load custom vocab created from Tokenizer library
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
Bert ["iuliaturc/bert_uncased_L-2_H-128_A-2"]
https://huggingface.co/iuliaturc/bert_uncased_L-2_H-128_A-2

Language I am using the model on (English, Chinese ...):
English

The problem arises when using:
my own modified scripts: (give details below)



The tasks I am working on is:
my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
see comments in the code 

1. Train model and tokenizer on new tokenizers library 
2. try to add new tokens to tokenizer
3. Resize token embeddings for model
 

### Problem 1 : 
If Loading with AutoTokenizer, can not use BertTokenizerFast but able to use custom vocabs

4. save tokenizer using save_tokenizer
5. Load using  AutoTokenizer

### Problem 2:
If Loading with BertTokenizerFast, can use the functioanlity of BertTokenizerFast but losed the custom vocabs

4. save tokenizer using save_tokenizer
5. Load using  BertTokenizerFast

```python


# Loaded Model and Tokenizer Default 
tokenizer = BertTokenizerFast.from_pretrained("""iuliaturc/bert_uncased_L-2_H-128_A-2""")
model = AutoModel.from_pretrained("""iuliaturc/bert_uncased_L-2_H-128_A-2""")

# Added Custom vocab to the tokenizer

def add_vocab_to_model(df, model, tokenizer, old_vocab, vocab_size=30000):
    """Adds new vocab to tokenizer and randomly initialises rows for new vocab in the model"""
    PATH = Path('./lm_data')
    PATH.mkdir(exist_ok=True)

    df.dropna(inplace=True)
    lm_text = ' '.join(df['text'])
    lm_text = lm_text.replace("■","")
    lm_text = re.sub(r'[^\x00-\x7F]+',' ', lm_text)
    lm_text = re.sub(r'[^\u0000-\u007F]+',' ', lm_text)

    with open('./lm_data/data.txt', mode='w') as f:
        f.write(lm_text)

    paths = [str(x) for x in Path("./lm_data/").glob("**/*.txt")]

    # Initialize a tokenizer
    tokenizer_new = BertWordPieceTokenizer(old_vocab, lowercase=True)

    # Customize training
    tokenizer_new.train(files=paths, vocab_size=vocab_size, min_frequency=20)
    tokenizer_new.save(".", "./lm_data/new")

    new_vocab = open('./lm_data/new-vocab.txt', 'r').read().split('\n')
    new_vocab.remove('')

    print('Adding new tokens to vocab')
    n_orig_tokens = len(tokenizer)
    tokenizer.add_tokens(new_vocab)
    print('Original no. of tokens: %s'%n_orig_tokens)
    print('Final no. of tokens: %s'%len(tokenizer))

    print('Initialised random emb for new tokens')
    model.resize_token_embeddings(len(tokenizer))

    return model, tokenizer


# Just needs to pass a dataframe which is having a column with name "text"
model1, tokenizer1 = add_vocab_to_model(dataframe, model, tokenizer, 'bert-base-uncased-vocab.txt', vocab_size=10000)
#Adding new tokens to vocab
#Original no. of tokens: 30522
#Final no. of tokens: 34340
#Initialised random emb for new tokens


print(len(tokenizer1))
# Got the result : 34340
#Finalized Result

print(type(tokenizer1))
# transformers.tokenization_bert.BertTokenizerFast
# Final Tokenizer


# Problem 1 : 
# If Loading with AutoTokenizer, can not use BertTokenizerFast but able to use custom vocabs


# Saved and Loaded Model Again
tokenizer1.save_pretrained("./tokenizer")
tokenizer2 = AutoTokenizer.from_pretrained("./tokenizer")

print(len(tokenizer2)
# Got 34340
# Which is Correct.

print(type(tokenizer2))
# transformers.tokenization_bert.BertTokenizer
# Not able to load BertTokenizerFast defaulting back to BertTokenizer


# Problem 2:
# If Loading with BertTokenizerFast, can use the functioanlity of BertTokenizerFast but losed the custom vocabs


# Saved and Loaded Model Again
tokenizer1.save_pretrained("./tokenizer")
tokenizer3 = BertTokenizerFast.from_pretrained("./tokenizer")

print(len(tokenizer3)
# Got 30522
# Losing the added custom vocabs

print(type(tokenizer3))
# transformers.tokenization_bert.BertTokenizerFast
#which is correct

```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->




## Expected behavior
1. If Loading tokenizer using AutoTokenizer it should load the BertTokenizerFast 
2. If Loading tokenizer using BertTokenizerFast, it should maintain custom vocabs


<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.9.1
- Platform: Windows
- Python version:  3.7.7
- PyTorch version (GPU?): 1.5.1 
- Tensorflow version (GPU?): NA
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

Hi,
Can anyone from Huggingface team look into this?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Similar to #6025 <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
