Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, NLP.
baseline. This is the first successful pre-trained NLP model which does not use discrete units (words, subwords, characters or bytes) as the training target. Instead, the model predicts a contextualized latent representation. 2.4. Ablation Studies Layer-averaged Targets One of the main differences