CLIPTokenizer behaves inconsistently depending on whether ftfy is installed or not
### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.22.1
- Platform: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.14
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.12.1+cu113 (False)
- Tensorflow version (GPU?): 2.8.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


### Who can help?

@patil-suraj

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run following code without ftfy installed.

```py
from transformers import CLIPTokenizer
tokenizer = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
tokenizer("résumé") # {'input_ids': [49406, 15077, 49407], 'attention_mask': [1, 1, 1]}
```

2. Run following code with ftfy installed.

```py
from transformers import CLIPTokenizer
tokenizer = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
tokenizer("résumé") # {'input_ids': [49406, 29106, 7054, 4166, 49407], 'attention_mask': [1, 1, 1, 1, 1]}
```

### Expected behavior

They should work consistently.
This happens because `BasicTokenizer`, which is [used as](https://github.com/huggingface/transformers/blob/main/src/transformers/models/clip/tokenization_clip.py#L159) fallback text fix function, [strips accents if `do_lower_case=True`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bert/tokenization_bert.py#L174-L176).

We may fix this by explicitly set `strip_accents` to `False`; [ViT/L-14 tokenizer](https://huggingface.co/openai/clip-vit-large-patch14/raw/main/tokenizer.json) includes vocabs with accents, so I think stripping accents should not be done.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.