Dino, Computer Vision, Transformers, Ssl, Deep Learning.
used. The student model(small) tries to imitate the same as the larger teacher model keeping performance fixed. Cross-Entropyloss is used for student-teacher feature similarities keeping the VIT model common in both for Image level learnings. Random crops of the same image are used for Image level