Hi,
I do have a demo notebook on using the Trainer for fine-tuning the Vision Transformer here: Transformers-Tutorials/Fine_tuning_the_Vision_Transformer_on_CIFAR_10_with_the_🤗_Trainer.ipynb at master · NielsRogge/Transformers-Tutorials · GitHub 8.
ViT doesn’t expect input_ids and attention_mask as input, but pixel_values instead. Note that we will add support for attention_mask in the future.