Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, Distillation.
key encoder to 0.999. A small temperature value τ: (less than one) is used so that the model focuses mainly on transferring the relationships from the close neighborhoods of the query rather than faraway points. τ=0.04. Ours-2q: the teacher and student embeddings are decoupled, so used a separate