clean_up_tokenization_spaces=False does not behave correctly in AutoTokenizer's decode function 
input example:
```python
tok_ids = [ 101,  2632,  1011,  7110,  1010,  2142,  5424, 14041,  2727,  1011,  2260,  1011,  5757,   102,   0]
```

if we do decode
```python
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokenizer.decode(tok_ids, clean_up_tokenization_spaces=False)

# output is 
'[CLS] al - ain , united arab emirates 1996 - 12 - 06 [SEP] [PAD]'
# we can see here the ',' is separated from word 'ain'
```

if we do decode
```python
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
tokenizer.decode(tok_id, clean_up_tokenization_spaces=False)

# output is
'[CLS] al - ain, united arab emirates 1996 - 12 - 06 [SEP] [PAD]'
# the space between ',' and 'ain' is removed although we set clean_up_tokenization_spaces as False
```

following the above question, I find another inconsistent behavior:

```python
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokenizer.decode(tokenizer.encode("al-ain"))
# I expect decode result is "al-ain" but get "al - ain", so spaces are inserted
```

Is this designed as encoding and decoding the same word can lead to different output or it is a bug?

more precisely, if we do
```python
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokenizer.tokenize("al-ain")
# output is ['al', '-', 'ain']
```
but I expect it generates subwords not inserting spaces.
cc @SaulLu <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>@SaulLu is this issue resolved or WIP ?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>Hi @bugface,

> Is this designed as encoding and decoding the same word can lead to different output or it is a bug?

Short answer: yes :smiley: 

In general, the `decode` method is a best-effort method because in the `text -> encode -> sequence of ids` operation there may be some loss of information (and this is the case for Bert's tokenizer by design). In general this is not a big deal as Bert is not a generative model and therefore does not need to decode a purely generated id sequence (I'd be very interested to know what use case you use decode for - as this is the subject of quite a few issues on our side! :pray: )

Also regarding the `clean_up_tokenization_spaces`, this argument targets specific cases as you can see in the method used to perform this cleaning:
https://github.com/huggingface/transformers/blob/f394a2a50d8729cd1ca9b368e330ec50664c3292/src/transformers/tokenization_utils_base.py#L3392-L3415

IMO, the `-` case is less "common" than the one listed in the current cleaning script.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.