Skip `test_disk_offload` for `WhisperModelTest`
# What does this PR do?

Since #22486, `WhisperModelTest.test_disk_offload` to fail. I just blindly skip this test and guess it is ok.....?

_The documentation is not available anymore as the PR was closed or merged._<|||||>I am not able to use `openai/whisper-base` as I get GPU OOM. When I tried to use `openai/whisper-tiny.en`, I have a hard time to change the parameters to get a working input dict for the model.

If I just changed the values in model tester to get a larger model (but random), I get different kinds of error like `IndexError: list index out of range` in `dispatch_model` or `RuntimeError: Tensor on device meta is not on the expected device cuda:0!` in model forward. 

But even if I revert the change in #22486, the above attempts to use larger (fake) models still have the same issue. I guess we will have to look into this.<|||||>Convert to draft to avoid being merged.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.