QUESTION: How to perform 2D interpolation of pre-trained position embeddings for fine-tuning on Vision Transformers?
Hi all,

I know that in examples for fine-tuning on the pre-trained ViT, there is a ViTFeatureExtractor that can upsample images from CIFAR10 -> 224x224 to match the image resolution that that model is pre-trained on. However in the paper, the authors suggest to perform 2D Interpolation on the pre-trained position embeddings. I saw here in the TIMM source code (https://github.com/rwightman/pytorch-image-models/blob/54e90e82a5a6367d468e4f6dd5982715e4e20a72/timm/models/vision_transformer.py#L481) there is a method to handle this interpolation. Is there a similar method / parameter that I can use in the HuggingFace ViT to enable this interpolation behavior, in lieu of upsampling the CIFAR10 32x32 to match 224x224?
Hi,

See #12167