Deberta Tokenizer convert_ids_to_tokens() is not giving expected results
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.0
- Platform: Colab
- Python version: 3.9
- PyTorch version (GPU?): No
- Tensorflow version (GPU?): No
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information
I am using Deberta Tokenizer. `convert_ids_to_tokens()` of the tokenizer is not working fine. 

The problem arises when using:
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset

## To reproduce

Steps to reproduce the behavior:

1. Get Debrta Tokenizer 
```python
from transformers import DebertaTokenizer
deberta_tokenizer = DebertaTokenizer.from_pretrained('microsoft/deberta-base')
```
2. Encode Some Example Using Tokenizer
```python
example = "Hi I am Bhadresh. I found an issue in Deberta Tokenizer"
encoded_example = distilbert_tokenizer.encode(example)
```
3. Convert Ids to tokens:
```python
distilbert_tokenizer.convert_ids_to_tokens(encoded_example )
"""
Output: ['[CLS]', '17250', '314', '716', '16581', '324', '3447', '13', '314', '1043', '281', '2071', '287', '1024', '4835', '64', '29130', '7509', '[SEP]']
"""
```
[Colab Link For Reproducing](https://github.com/bhadreshpsavani/UnderstandingNLP/blob/master/DebertaTokenizerIssue.ipynb)
## Expected behavior

It should return some tokens like this
```
['[CLS]', 'hi', 'i', 'am', 'b', '##had', '##resh', '.', 'i', 'found', 'an', 'issue', 'in', 'de', '##bert', '##a', 'token', '##izer', '[SEP]']
```
Not just convert an integer to string like the current behavior

#### Tagging SMEs for help: 
@n1t0, @LysandreJik
It seems expected behavior but something is still not right with this tokenizer<|||||>Seems like they have not implemented a decoder for the tokenizer. I will have a look at it.<|||||>It might be expected behaviour because it is based on GPT2 tokenizer and it is also having similar results
<|||||>That is not true:
```
from transformers import GPT2Tokenizer
t = GPT2Tokenizer.from_pretrained('gpt2')
encoded = t("Hi I am Bhadresh. I found an issue in Deberta Tokenizer")
t.convert_ids_to_tokens(encoded['input_ids'])
```
['Hi',
 'ĠI',
 'Ġam',
 'ĠBh',
 'ad',
 'resh',
 '.',
 'ĠI',
 'Ġfound',
 'Ġan',
 'Ġissue',
 'Ġin',
 'ĠDe',
 'bert',
 'a',
 'ĠToken',
 'izer']<|||||>Ya, you are right!, Something is missing in the implementation I can't figure out what!

I try to convert the SQUAD2 dataset in to feature using the SQAUD.py file in data preprocessing. 
After Conversion when I decode the input id it is returning context like this

`Who are you?IamBhadresh`

I mean in context space is not considered!

The ConvertExampletoFeature uses `convert_ids_to_tokens` internally I suspect that was creating issue<|||||>You can convert them back with the following code:
```
from transformers import DebertaTokenizer
t = DebertaTokenizer.from_pretrained('microsoft/deberta-base')
example = "Hi I am Bhadresh. I found an issue in Deberta Tokenizer"

encoded_example = t.encode(example)

[t.gpt2_tokenizer.decode([t.gpt2_tokenizer.sym(id)]) if t.gpt2_tokenizer.sym(id) not in t.all_special_tokens else t.gpt2_tokenizer.sym(id) for id in encoded_example]
```
Output:
```
['[CLS]',
 'Hi',
 ' I',
 ' am',
 ' Bh',
 'ad',
 'resh',
 '.',
 ' I',
 ' found',
 ' an',
 ' issue',
 ' in',
 ' De',
 'bert',
 'a',
 ' Token',
 'izer',
 '[SEP]']
```

After some digging into the code, I am actually not sure if I should create a patch for it or not. I think with a patch we can **probably** also remove the method [download_asset](https://github.com/huggingface/transformers/blob/cdd31b4de4b446ccff9428d14fbeb45c4d96c608/src/transformers/models/deberta/tokenization_deberta.py#L224) and refactor the [load_vocab](https://github.com/huggingface/transformers/blob/cdd31b4de4b446ccff9428d14fbeb45c4d96c608/src/transformers/models/deberta/tokenization_deberta.py#L270) method. 

I am not sure if this was discussed before but when we create the required files from the `bpe_encoder.bin`, we could probably get rid of the [GPT2Tokenizer](https://github.com/huggingface/transformers/blob/cdd31b4de4b446ccff9428d14fbeb45c4d96c608/src/transformers/models/deberta/tokenization_deberta.py#L301) class in tokenization_deberta.py and the DebertaTokenizer could inherit directly from the GPT2Tokenizer (like the RobertaTokenizer).

I will leave it to @LysandreJik and @BigBird01 to decide what to do with it.  <|||||>@BigBird01, would you be open to having the `DebertaTokenizer` inheriting directly from the GPT-2 tokenizer as @cronoik proposes? It would prevent such cases like the one mentioned in this issue from happening.<|||||>Yes. Let’s do it this way.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Lysandre Debut <notifications@github.com>
Sent: Monday, February 22, 2021 5:57:14 AM
To: huggingface/transformers <transformers@noreply.github.com>
Cc: Pengcheng He <biglm100@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [huggingface/transformers] Deberta Tokenizer convert_ids_to_tokens() is not giving expected results (#10258)


@BigBird01<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FBigBird01&data=04%7C01%7CPengcheng.H%40microsoft.com%7C25586e57796247f3a7f108d8d739c427%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637495990401189739%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2WajMgFu%2BFniyQviBr3x6%2BcK0dp4q4k2mXPctOA5Ldo%3D&reserved=0>, would you be open to having the DebertaTokenizer inheriting directly from the GPT-2 tokenizer as @cronoik<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcronoik&data=04%7C01%7CPengcheng.H%40microsoft.com%7C25586e57796247f3a7f108d8d739c427%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637495990401189739%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=HE78YNCso1ULWJoBDEOahmmpmgdE7%2Bq7jrSBwpq1%2FiU%3D&reserved=0> proposes? It would prevent such cases like the one mentioned in this issue from happening.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhuggingface%2Ftransformers%2Fissues%2F10258%23issuecomment-783393615&data=04%7C01%7CPengcheng.H%40microsoft.com%7C25586e57796247f3a7f108d8d739c427%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637495990401199701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4j7yOLhgDl83UAGBcxomF%2F3iBjpQScMRqFh%2BneP64ro%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJDNDRU4H6NOWYGMYNP7BMTTAJPDVANCNFSM4X2GBZKQ&data=04%7C01%7CPengcheng.H%40microsoft.com%7C25586e57796247f3a7f108d8d739c427%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637495990401199701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GhMxfK7jJhigF9AloayWdcB9Z3lA6nxfxki%2F4Gef4TE%3D&reserved=0>.
<|||||>@cronoik do you want to take a stab at it?<|||||>Yes.
@LysandreJik 