Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, Distillation.
a random set of anchor points (a) for both teacher and student, their corresponding embeddings are: Given a query image qi and its embeddings tqi for teacher and sqi for student, the pairwise similarity is estimated. For the teacher, the probability of the i-th query for the j-th anchor point is: