tokenizer.add_tokens not working
## 🐛 Bug

<!-- Important information -->
I tried to add new tokens in vocabulary using tokenizer.add_tokens() and then called model() according to the code given in `BertForMaskedLM` class definition. The code is given below: 
```

from transformers import BertForMaskedLM, BertTokenizer
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMaskedLM.from_pretrained('bert-base-uncased')

tokenizer.add_tokens(['[SPECIAL_TOKEN_1]', '[SPECIAL_TOKEN_2]'])
model.resize_token_embeddings(len(tokenizer))

input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, masked_lm_labels=input_ids)
```

I get the following error:
`RuntimeError: The size of tensor a (30524) must match the size of tensor b (30522) at non-singleton dimension 2`

I believe this was fixed recently. Could you please try installing from source `pip install git+https://github.com/huggingface/transformers` and let me know if it fixes the bug?<|||||>I executed the above command and it worked.
Thanks<|||||>Glad it worked.<|||||>I found a similar bug even with the latest version built from source. 
After adding new tokens, if I use `len(tokenizer)`, I can see that the total number of tokens has increased. However, if I use `tokenizer.vocab_size`, the size was still the number before adding new tokens. If I save the vocab using `tokenizer.save_vocabulary("./")`, the generated vocab.json file does not contain the new added tokens. 
<|||||>Hello! This is not an error. Your added tokens are in `added_tokens.json`.<|||||>> Hello! This is not an error. Your added tokens are in `added_tokens.json`.

Thanks for your reply! However, when I save the RobtertaTokenizer, there are only vocab.json and merge.txt. I can't find the file added_tokens.json.<|||||>Which version of transformers are you using? In the latest version:

```py
>>> from transformers import RobertaTokenizer
>>> tok = RobertaTokenizer.from_pretrained("roberta-base")
>>> tok.add_tokens(["lingjzhu", "LysandreJik"])
2
>>> tok.save_pretrained("here")
('here/vocab.json', 'here/merges.txt', 'here/special_tokens_map.json', 'here/added_tokens.json')
```

When inspecting `here/added_tokens.json`:

```
{"lingjzhu": 50265, "LysandreJik": 50266}
```<|||||>Thanks for your comments! I recompiled the package from the source and it is working now. I am sorry  for the negligence!<|||||>I just want to add a comment about the tokenizer. The function `tokenizer.save_vocabulary()` will not save the added tokens even in the latest version. This was my original error. But `tokenizer.save_pretrained()` will solve the problem. <|||||>@lingjzhu, that makes sense, `save_vocabulary` saves the vocabulary. The entire tokenizer (with the special tokens, with the added tokens, with the special added tokens) needs to be saved using `save_pretrained`, as you've said.

The difference is explicitely mentioned in the [documentation](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.save_pretrained).<|||||>> Which version of transformers are you using? In the latest version:
> 
> ```python
> >>> from transformers import RobertaTokenizer
> >>> tok = RobertaTokenizer.from_pretrained("roberta-base")
> >>> tok.add_tokens(["lingjzhu", "LysandreJik"])
> 2
> >>> tok.save_pretrained("here")
> ('here/vocab.json', 'here/merges.txt', 'here/special_tokens_map.json', 'here/added_tokens.json')
> ```
> 
> When inspecting `here/added_tokens.json`:
> 
> ```
> {"lingjzhu": 50265, "LysandreJik": 50266}
> ```
Hi, since I append the origin vocab.json according to the added_tokens.json file, and the vocab size and tokenizer length both added from 21128 to 21300. However, convert_tokens_to_ids() function seems that referenced the origin vocab.json with 21128 length, is there any solutions to use both origin and added tokens to apply the convert_tokens_to_ids() function?
