special token inconsistency for [UNK] token
## Environment info
     
- `transformers` version: 3.0.2
- Platform: Linux-4.15.0-108-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9

### Who can help
 tokenizers: @mfuntowicz

## Information

Bert tokenizer treats [UNK] token as **special** during `tokenizer.convert_ids_to_tokens(...,skip_special_tokens=True) ` 
while `tokenizer(...,return_special_tokens_mask=True)` doesn't (for obvious reasons).
I think it would be better to preserve [UNK] tokens in `convert_ids_to_tokens` for consistency of the term "**special token**".

## To reproduce

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
# fast tokenizer also has the same problem
# tokenizer = AutoTokenizer.from_pretrained("bert-large-cased",use_fast=True)
text="sentence 한 셔 word"
print('tokens:',tokenizer.tokenize(text))

tokens=tokenizer(text,return_special_tokens_mask=True)
print("input_ids:",tokens["input_ids"])
print("special_token_mask:",tokens["special_tokens_mask"])
no_special=tokenizer.convert_ids_to_tokens(tokens["input_ids"], skip_special_tokens=True)
special=tokenizer.convert_ids_to_tokens(tokens["input_ids"])
print('tokens from ids (skip special): ',no_special)
print('tokens from ids (skip special): ',special)

print('special tokens',tokenizer.all_special_tokens)
```

## Expected behavior

Also I think keeping [UNK] would be a better behavior, as `convert_ids_to_tokens` is used in inference pipelines and using skip_special_token for getting rid of [CLS][SEP][PAD] tokens leads to unintended loss of [UNK] tokens, which is important.

Related: https://github.com/huggingface/transformers/issues/4391

I think that's reasonable, the point of `skip_special_tokens` isn't to skip unknown tokens. cf @mfuntowicz @thomwolf @n1t0 <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
