Vision Transformer (ViT) feature vector example (not classification)
# 🚀 Feature request


## Motivation

I would like to see an example of using Vision Transformer just for feature extraction, get the feature vector, before the classification part of the neural network as can be done with tensorflow hub
https://www.tensorflow.org/hub/common_signatures/images?hl=en#image_feature_vector

In HuggingFace Transformers, models typically output a dictionary. You can access the feature vector by getting the `pooler_output` key of that dictionary (assuming you're using `ViTModel`). It's a tensor of shape `(batch_size, hidden_size)`, so in case you're only providing a single image, and you're using the base-sized model, this will be a tensor of shape `(1, 768)`. 

Here's an example:

```
from transformers import ViTFeatureExtractor, ViTModel
from PIL import Image
import requests

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = ViTFeatureExtractor.from_pretrained('google/vit-base-patch16-224-in21k')
model = ViTModel.from_pretrained('google/vit-base-patch16-224-in21k')

inputs = feature_extractor(images=image, return_tensors="pt")
outputs = model(**inputs)
feature_vector = outputs.pooler_output
```<|||||>thank you, that is what i was looking for !
thank you<|||||>@NielsRogge is it always `pooler_output` key that contains the feature vector for all image transformers (such as CLIP, DeiT, VisualBERT, DETR)?