Inconsistent ProphetNet Tokenization
## Environment info

- `transformers` version: 4.4.2
- Platform: Linux-5.4.0-1041-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.10
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

@LysandreJik @sgugger @patrickvonplaten 

## Information

Model I am using (Bert, XLNet ...): ProphetNet

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

I am using ProphetNet for some question generation work. I ran into a bug where I sampled from ProphetNet to generate a question generation using the `generate` function. I then took the outputted question and tokenized it and saw that the token_ids produced were different than the ones returned by the `generate` function. Here is a sample of the example I found to be inconsistent.

```Python
>>> from transformers import ProphetNetTokenizer
>>> tokenizer = ProphetNetTokenizer.from_pretrained("microsoft/prophetnet-large-uncased-squad-qg")
>>> tokenizer.decode([2054, 2828, 1997, 6922, 2003, 1037, 12702, 5054, 17119, 1037, 2828, 1997, 1029, 102])
'what type of component is a microsencer a type of? [SEP]'
>>> tokenizer.decode([2054, 2828, 1997, 6922, 2003, 1037, 12702, 3366, 5897, 2099, 1037, 2828, 1997, 1029, 102])
'what type of component is a microsencer a type of? [SEP]'
>>> tokenizer.decode([3366, 5897, 2099])
'##sencer'
>>> tokenizer.decode([5054, 17119])
'##sencer'
>>> tokenizer.decode([5054, 17119]) == tokenizer.decode([3366, 5897, 2099])
True
>>> tokenizer("what type of component is a microsencer a type of? [SEP]")["input_ids"]
[2054, 2828, 1997, 6922, 2003, 1037, 12702, 5054, 17119, 1037, 2828, 1997, 1029, 102, 102]
```

It seems odd there are these two different sets of token_ids that reproduce the same sentence. I am having trouble reproducing the second sequence of tokens. I was training the model when it produced this second sequence through sampling but I didn't save the model at that exact time.

## Expected behavior

I would expect the tokenizer to always return the same sequence of tokens. I am not too familiar with how the tokenizers work, so I might be wrong in this assumption. Is this behavior expected and should I just try to avoid re-tokenizing whenever possible?

> I would expect the tokenizer to always return the same sequence of tokens. I am not too familiar with how the tokenizers work, so I might be wrong in this assumption. Is this behavior expected and should I just try to avoid re-tokenizing whenever possible?

Why should the tokenizer do that? You give him some ids and expect the tokenizer to convert them into a string based on its dictionary. When you look at the ids and their respective tokens you will see that the tokenizer is working correctly:
```
[valueKey[x] for x in [3366,5897,2099]]
['##se', '##nce', '##r']
```
```
[valueKey[x] for x in [5054,17119]]
['##sen', '##cer']
```
Keep in mind that each of those tokens is needed to build other words even when they seem like a duplicate for the word `microsencer`.


<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.