Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, Distillation.
memory bank (queue) for each. Ours-1q: uses the teacher’s anchor points in calculating the similarity for the student model. Caching teacher: Calculating the embeddings for the teacher is expensive in terms of both computation and memory. Caching teacher makes training faster. A drawback is that we