Deep Learning, Artificial Intelligence, Self Supervised Learning, Image Classification, Unsupervised Learning.
Distillation, a single augmented image (instead of the 2-view setup) is used in combination with a simple regression loss to predict the representations. Given an augmented input image x with clustering id k, it is fed into the Distillation model to produce the projection-head output z. Thus, we