Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, NLP.
are left unchanged and 10% are replaced by randomly selected vocabulary token. 1.3. Training Targets The representations to predict are contextualized representations. This is an important difference to BERT, wav2vec 2.0 or BEiT, MAE, SimMIM, and MaskFeat, which predict targets lacking contextual