Paper Review, Deep Learning, Self Supervised Learning, Computer Vision, Pretrained Model.
improved when transitioning from training on ImageNet-1K (IN1K) to ImageNet-22K (IN22K). The study also found that I-JEPA benefits from larger model size when pretraining on larger datasets such as IN22K. For example, pretraining a larger ViT-G/16 model improved the downstream performance on image