Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, NLP.
of our method compared to BYOL is the use of targets which are based on averaging multiple layers from the teacher network. Targets based on multiple layers improves over using only the top layer (K = 1) for all modalities. Target Contextualization Teacher representations are based on