CI Tests are failing in "run_tests_pipelines_tf"
### System Info

```shell
CI setup for `run_tests_pipelines_tf`:

run_tests_pipelines_tf:
        working_directory: ~/transformers
        docker:
            - image: circleci/python:3.7
        environment:
            OMP_NUM_THREADS: 1
            RUN_PIPELINE_TESTS: yes
            TRANSFORMERS_IS_CI: yes
        resource_class: xlarge
        parallelism: 1
        steps:
            - checkout
            - restore_cache:
                  keys:
                      - v0.4-tf-{{ checksum "setup.py" }}
                      - v0.4-{{ checksum "setup.py" }}
            - run: pip install --upgrade pip
            - run: pip install .[sklearn,tf-cpu,testing,sentencepiece]
            - run: pip install tensorflow_probability
            - save_cache:
                  key: v0.4-tf-{{ checksum "setup.py" }}
                  paths:
                      - '~/.cache/pip'
            - run: python utils/tests_fetcher.py | tee test_preparation.txt
            - store_artifacts:
                  path: ~/transformers/test_preparation.txt
            - run: |
                  if [ -f test_list.txt ]; then
                    python -m pytest -n 8 --max-worker-restart=0 --dist=loadfile -rA -s --make-reports=tests_pipelines_tf $(cat test_list.txt) -m is_pipeline_test | tee tests_output.txt
                  fi
            - store_artifacts:
                  path: ~/transformers/tests_output.txt
            - store_artifacts:
                  path: ~/transformers/reports
```


### Who can help?

@ydshieh , @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Following CI tests are failing in "run_tests_pipelines_tf" for PR #17623  when no changes are done with respect to TF or speech functionalities

```
FAILED tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_chunking_fast
FAILED tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_return_timestamps_ctc_fast
FAILED tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_small_model_pt
===== 3 failed, 680 passed, 578 skipped, 270 warnings in 222.67s (0:03:42) =====
```

```=================================== FAILURES ===================================
__________ AutomaticSpeechRecognitionPipelineTests.test_chunking_fast __________
[gw1] linux -- Python 3.7.12 /usr/local/bin/python

self = Audio(sampling_rate=16000, mono=True, decode=True, id=None)
value = '/home/circleci/.cache/huggingface/datasets/downloads/extracted/06793a6d1707e1987473fd67ba38f9156c15e5a8d2956a4fff1d9690877b20a8/dev_clean/1272/128104/1272-128104-0000.flac'

    def encode_example(self, value: Union[str, dict]) -> dict:
        """Encode example into a format for Arrow.
    
        Args:
            value (:obj:`str` or :obj:`dict`): Data passed as input to Audio feature.
    
        Returns:
            :obj:`dict`
        """
        try:
>           import soundfile as sf  # soundfile is a dependency of librosa, needed to decode audio files.
E           ModuleNotFoundError: No module named 'soundfile'

../.local/lib/python3.7/site-packages/datasets/features/audio.py:83: ModuleNotFoundError
```

```
_________ AutomaticSpeechRecognitionPipelineTests.test_small_model_pt __________
[gw3] linux -- Python 3.7.12 /usr/local/bin/python

self = <tests.pipelines.test_pipelines_automatic_speech_recognition.AutomaticSpeechRecognitionPipelineTests testMethod=test_small_model_pt>

    @require_torch
    def test_small_model_pt(self):
        speech_recognizer = pipeline(
            task="automatic-speech-recognition",
            model="facebook/s2t-small-mustc-en-fr-st",
            tokenizer="facebook/s2t-small-mustc-en-fr-st",
>           framework="pt",
        )

tests/pipelines/test_pipelines_automatic_speech_recognition.py:136: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/transformers/pipelines/__init__.py:638: in pipeline
    feature_extractor, revision=revision, _from_pipeline=task, **model_kwargs
src/transformers/models/auto/feature_extraction_auto.py:326: in from_pretrained
    return feature_extractor_class.from_dict(config_dict, **kwargs)
src/transformers/utils/import_utils.py:809: in __getattr__
    requires_backends(cls, cls._backends)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <class 'transformers.utils.dummy_speech_objects.Speech2TextFeatureExtractor'>
backends = ['speech']

    def requires_backends(obj, backends):
        if not isinstance(backends, (list, tuple)):
            backends = [backends]
    
        name = obj.__name__ if hasattr(obj, "__name__") else obj.__class__.__name__
        checks = (BACKENDS_MAPPING[backend] for backend in backends)
        failed = [msg.format(name) for available, msg in checks if not available()]
        if failed:
>           raise ImportError("".join(failed))
E           ImportError: 
E           Speech2TextFeatureExtractor requires the torchaudio library but it was not found in your environment. You can install it with pip:
E           `pip install torchaudio
```

### Expected behavior

```shell
No tests fail
```
