Default rescale for ImageProcessing
### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-5.4.0-1096-gcp-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.5
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Distributed

### Who can help?

@amyeroberts Is there any reason that the default value for `do_rescale` is set to `True`? Does it mess with current ViT pre-trained model such as Google-ViT? I am trying to pre-train my model and saw that there is this discrepancy that arguably could impact accuracy. I am just trying to figure out the rationale behind this.

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. from transformers import AutoFeatureExtractor
2. Autofeature_extractor = AutoFeatureExtractor.from_pretrained("google/vit-base-patch16-224")
3. Autofeature_extractor

```
ViTImageProcessor {
  "do_normalize": true,
  "do_rescale": true,
  "do_resize": true,
  "image_mean": [
    0.5,
    0.5,
    0.5
  ],
  "image_processor_type": "ViTImageProcessor",
  "image_std": [
    0.5,
    0.5,
    0.5
  ],
  "resample": 2,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "height": 224,
    "width": 224
  }
}
```

### Expected behavior

```
ViTImageProcessor {
  "do_normalize": true,
  "do_resize": true,
  "image_mean": [
    0.5,
    0.5,
    0.5
  ],
  "image_processor_type": "ViTImageProcessor",
  "image_std": [
    0.5,
    0.5,
    0.5
  ],
  "resample": 2,
  "size": {
    "height": 224,
    "width": 224
  }
}
```
@yazdanbakhsh  Thanks for creating the issue and putting in so much detail. 

If I've understood correctly, your question is about differences in the configuration seen on the hub e.g. [here](https://huggingface.co/google/vit-base-patch16-224/blob/main/preprocessor_config.json) and the object representation when it's loaded and whether this affects training your model. I'll answer for this, but let me know if there's anything I've missed. 

TLDR; This will not affect training your model unless your input images are numpy arrays and `do_resize=False`.  

The feature extractors have recently been deprecated in place of image processors. The feature extractors now act as an alias for the image processor, and using `AutoFeatureExtractor` will load an image processor under the hood.  

At the moment, the previous feature extractor configurations are loaded and converted to the equivalent image processor configuration. For example, you'll notice that [`size` in `preprocessor_config.json`](https://huggingface.co/google/vit-base-patch16-224/blob/main/preprocessor_config.json) is an int `224`, whereas it's a dictionary in the image processor. In time, these configurations will be updated on the hub. 

With respect to `do_rescale`, this flag has been added in order to separate the concerns of certain processing logic. [Rescaling still happened in the old feature extractors](https://github.com/huggingface/transformers/blob/4fd89e49788f60b021b4e2c578a1fb12f1e900e4/src/transformers/image_utils.py#LL380C7-L380C7), this just makes the step more explicit and controllable by the user. Previously, images would have their pixel values divided by 255 if `do_normalize=True` and the input image was a `PIL.Image.Image` or `do_resize=True`, however they wouldn't be rescaled if the input was a numpy array and `do_resize=False`. This ensures consistent rescaling behaviour, regardless of the input type. As a result, there may be differences in the resulting images between the old feature extractors and new image processors if your input images are numpy arrays and `do_resize=False`, however, the resulting image should be consistent across input types for all flag combinations with the new image processors. <|||||>@amyeroberts Thank you so much for the detailed explanation. That all make sense to me. I just wanted to share this to ensure that this is an expected behavior from the implementation. Based on your explanation, I think we should be able to close this issue.