Run ViT examples on PyTorch XLA out of the box
Check if Vision Transformers (ViT) model and its example script are supported on PyTorch/XLA out of the box. If not, produce the code to bring the needed support.
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15599). All of your documentation changes will be reflected on that endpoint.<|||||>It turns out the model is mostly supported. 
I had to make a couple of change to successfully run the model on the TPU using PyTorch/XLA. Below is the command to repro:

```
python examples/pytorch/xla_spawn.py --num_cores 1 examples/pytorch/image-classification/run_image_classification.py   \
--dataset_name cats_vs_dogs     --output_dir ./cats_vs_dogs_outputs/  \
--remove_unused_columns False     --do_train     --do_eval  --learning_rate 2e-4 \
--num_train_epochs 5     --per_device_train_batch_size 32     --per_device_eval_batch_size 32 \
--logging_strategy steps     --logging_steps 10     --evaluation_strategy epoch     --save_strategy epoch  \
--load_best_model_at_end True     --save_total_limit 3     --seed 1337
```<|||||>Hi @sgugger. 

Basic questions: I just locally ran the code quality test. Not sure if I understand what it requires to correct the issue given the small size of my code change. How do you usually handle this issue?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>I dropped the ball here.
@steventk-g can please follow up on this PR and help land it in HF stack?