Problem at using CLIPFeatureExtractor from transformers.models.clip.feature_extraction_clip
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.17.0
- Platform: Ubuntu
- Python version: 3.7
- PyTorch version (GPU?): 1.7.0
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Models:

CLIPFeatureExtractor (which is used to provide feature to CLIPModel)

## Information

When I use CLIPFeatureExtractor(), (https://huggingface.co/transformers/v4.6.0/_modules/transformers/models/clip/feature_extraction_clip.html#CLIPFeatureExtractor) at the __call__ function, after images pass through resize, center_crop, and normalize, it is given to `BatchFeature` of `feature_extraction_utils` and then problem happens.

The problem happens inside `convert_to_tensors` function, inside the for loop starting from line 166, the code tries to change value to tensor, but sometimes the type of value is a list or something else, so it goes to except statement. So I had to write an extra snippet for the case when the value is list.


Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)

The tasks I am working on is:
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

`clip_feature_extractor = CLIPFeatureExtractor()`
`# images are list of images in numpy array or some other types such as PIL or tensor`
`clip_feature_extractor(images=images, return_tensors='pt')`
Hi,

> images are list of images in numpy array or some other types such as PIL or tensor
clip_feature_extractor(images=images, return_tensors='pt')

If you pass a list of images, make sure they are all of the same type.<|||||>Yes I did passed a list of images whose images are all in same type, but the problem happens<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>Can you elaborate, or provide a minimal reproducer?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>I'm also having this issue.  It only works if I pass in PIL images.  It does not work on tensors _or_ lists of tensors.