Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, NLP.
image classification, the output of the last Transformer block is mean-pooled and fed to a softmax-normalized classifier. For speech, fine-tuning regime of wav2vec 2.0 is used. 2. Results 2.1. Computer Vision Computer Vision ImageNet is used for pretraining. data2vec outperforms prior work with