[WHISPER] Update modeling tests
# What does this PR do?
Fixes the test after the update of the tokenizer, which added sufix and prefix tokens. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.<|||||>I agree, it is indeed better to just use `add_special_tokens=False`! Slipped my mind 😉 <|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.<|||||>Ping me again once when you think it's ready :) <|||||>Looks like we still need to add `add_special_tokens=False` to the tf test @ArthurZucker!<|||||>Yeah on it 🤗<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.<|||||>Ah there's small bug with this, the kwargs is passed to `self.pad` in the feature extractor. Gonna fix that<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20162). All of your documentation changes will be reflected on that endpoint.