Transformers Pipeline Error
When I tried to use pipeline from transformers I got this error. It happened when tensorflow was installed but pytorch was not.

```
from transformers import pipeline
pipe = pipeline(model="lysandre/tiny-vit-random")
```

Error:

```
>       pipe = pipeline(model="lysandre/tiny-vit-random")

test_external.py:248: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\venv\lib\site-packages\transformers\pipelines\__init__.py:541: in pipeline
    framework, model = infer_framework_load_model(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

model = 'lysandre/tiny-vit-random'
config = ViTConfig {
  "_name_or_path": "lysandre/tiny-vit-random",
  "architectures": [
    "ViTForImageClassification"
  ],
 ...m_channels": 3,
  "num_hidden_layers": 2,
  "patch_size": 16,
  "qkv_bias": true,
  "transformers_version": "4.16.2"
}

model_classes = {'pt': (), 'tf': ()}, task = 'image-classification'
framework = None
model_kwargs = {'_from_pipeline': 'image-classification', 'revision': None, 'use_auth_token': None}
class_tuple = (<class 'transformers.models.vit.modeling_tf_vit.TFViTForImageClassification'>,)
look_pt = False, look_tf = True
classes = [<class 'transformers.models.vit.modeling_tf_vit.TFViTForImageClassification'>]
architecture = 'ViTForImageClassification'
transformers_module = <module 'transformers' from 'F:\\SecondaryDownloads\\git_repos\\gradio\\venv\\lib\\site-packages\\transformers\\__init__.py'>
_class = <class 'transformers.models.vit.modeling_tf_vit.TFViTForImageClassification'>

    def infer_framework_load_model(
        model,
        config: AutoConfig,
        model_classes: Optional[Dict[str, Tuple[type]]] = None,
        task: Optional[str] = None,
        framework: Optional[str] = None,
        **model_kwargs
    ):
        """
        Select framework (TensorFlow or PyTorch) to use from the `model` passed. Returns a tuple (framework, model).
    
        If `model` is instantiated, this function will just infer the framework from the model class. Otherwise `model` is
        actually a checkpoint name and this method will try to instantiate it using `model_classes`. Since we don't want to
        instantiate the model twice, this model is returned for use by the pipeline.
    
        If both frameworks are installed and available for `model`, PyTorch is selected.
    
        Args:
            model (`str`, [`PreTrainedModel`] or [`TFPreTrainedModel`]):
                The model to infer the framework from. If `str`, a checkpoint name. The model to infer the framewrok from.
            config ([`AutoConfig`]):
                The config associated with the model to help using the correct class
            model_classes (dictionary `str` to `type`, *optional*):
                A mapping framework to class.
            task (`str`):
                The task defining which pipeline will be returned.
            model_kwargs:
                Additional dictionary of keyword arguments passed along to the model's `from_pretrained(...,
                **model_kwargs)` function.
    
        Returns:
            `Tuple`: A tuple framework, model.
        """
        if not is_tf_available() and not is_torch_available():
            raise RuntimeError(
                "At least one of TensorFlow 2.0 or PyTorch should be installed. "
                "To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ "
                "To install PyTorch, read the instructions at https://pytorch.org/."
            )
        if isinstance(model, str):
            model_kwargs["_from_pipeline"] = task
            class_tuple = ()
            look_pt = is_torch_available() and framework in {"pt", None}
            look_tf = is_tf_available() and framework in {"tf", None}
            if model_classes:
                if look_pt:
                    class_tuple = class_tuple + model_classes.get("pt", (AutoModel,))
                if look_tf:
                    class_tuple = class_tuple + model_classes.get("tf", (TFAutoModel,))
            if config.architectures:
                classes = []
                for architecture in config.architectures:
                    transformers_module = importlib.import_module("transformers")
                    if look_pt:
                        _class = getattr(transformers_module, architecture, None)
                        if _class is not None:
                            classes.append(_class)
                    if look_tf:
                        _class = getattr(transformers_module, f"TF{architecture}", None)
                        if _class is not None:
                            classes.append(_class)
                class_tuple = class_tuple + tuple(classes)
    
            if len(class_tuple) == 0:
                raise ValueError(f"Pipeline cannot infer suitable model classes from {model}")
    
            for model_class in class_tuple:
                kwargs = model_kwargs.copy()
                if framework == "pt" and model.endswith(".h5"):
                    kwargs["from_tf"] = True
                    logger.warning(
                        "Model might be a TensorFlow model (ending with `.h5`) but TensorFlow is not available. "
                        "Trying to load the model with PyTorch."
                    )
                elif framework == "tf" and model.endswith(".bin"):
                    kwargs["from_pt"] = True
                    logger.warning(
                        "Model might be a PyTorch model (ending with `.bin`) but PyTorch is not available. "
                        "Trying to load the model with Tensorflow."
                    )
    
                try:
                    model = model_class.from_pretrained(model, **kwargs)
                    if hasattr(model, "eval"):
                        model = model.eval()
                    # Stop loading on the first successful load.
                    break
                except (OSError, ValueError):
                    continue
    
            if isinstance(model, str):
>               raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
E               ValueError: Could not load model lysandre/tiny-vit-random with any of the following classes: (<class 'transformers.models.vit.modeling_tf_vit.TFViTForImageClassification'>,).

..\venv\lib\site-packages\transformers\pipelines\base.py:235: ValueError
```
Hey @FarukOzderim, `lysandre/tiny-vit-random` is a dummy model that should probably be deleted by now. What is the use-case you're trying to achieve with it?

It's currently used in the tests, but it should be removed.<|||||>I also use it in the tests within Gradio, please do not remove it :D
Just wanted to report the occurrence, I have no problem after installing torch.<|||||>I recommend using this one instead, which will be maintained forever: https://huggingface.co/hf-internal-testing/tiny-random-vit

Thanks for the error report!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.